ORIGINAL ARTICLE
Developing and Evaluating Composite Measures of Cancer Care Quality Cleo A. Samuel, PhD,* Alan M. Zaslavsky, PhD,w Mary Beth Landrum, PhD,w Karl Lorenz, MD, MSHS,zy and Nancy L. Keating, MD, MPHw8
Background: Composite measures are useful for distilling quality data into summary scores; yet, there has been limited use of composite measures for cancer care. Objective: Compare multiple approaches for generating cancer care composite measures and evaluate how well composite measures summarize dimensions of cancer care and predict survival. Study Design: We computed hospital-level rates for 13 colorectal, lung, and prostate cancer process measures in 59 Veterans Affairs hospitals. We computed 4 empirical-factor (based on an exploratory factor analysis), 3 cancer-specific (colorectal, lung, prostate care), and 3 care modality–specific (diagnosis/evaluation, surgical, nonsurgical treatments) composite measures. We assessed correlations among all composite measures and estimated all-cause survival for colon, rectal, non–small cell lung, and small cell lung cancers as a function of composite scores, adjusting for patient characteristics. Results: Four factors emerged from the factor analysis: nonsurgical treatment, surgical treatment, colorectal early diagnosis, and prostate treatment. We observed strong correlations (r) among composite measures comprised of similar process measures (r = 0.58–1.00, P < 0.0001), but not among composite measures reflecting different care dimensions. Composite measures were rarely associated with survival. Conclusions: The empirical-factor domains grouped measures variously by cancer type and care modality. The evidence did not
From the *Department of Health Policy and Management, Gillings School of Global Public Health, University of North Carolina at Chapel Hill, Chapel Hill, NC; wDepartment of Health Care Policy, Harvard Medical School, Boston, MA; zDivisions of General Internal Medicine and Palliative Care, Veterans Administration Greater Los Angeles Healthcare System; yGeffen School of Medicine, University of California Los Angeles, Los Angeles, CA; and 8Division of General Internal Medicine, Brigham and Women’s Hospital, Boston, MA. Supported by the Department of Veterans Affairs through the Office of Policy and Planning as part of an evaluation of oncology care. C.A.S.’s effort was supported by the Ruth L. Kirschstein National Research Service Award and the Harvard University Graduate Prize Fellowship. The authors declare no conflict of interest. Reprints: Nancy L. Keating, MD, MPH, Department of Health Care Policy, Harvard Medical School, 180 Longwood Avenue, Boston, MA 02115. E-mail: [email protected]. Supplemental Digital Content is available for this article. Direct URL citations appear in the printed text and are provided in the HTML and PDF versions of this article on the journal’s Website, www.lwwmedicalcare.com. Copyright r 2014 by Lippincott Williams & Wilkins ISSN: 0025-7079/15/5301-0054
54 | www.lww-medicalcare.com
support any single approach for generating cancer care composite measures. Weak associations across different care domains suggest that low-quality and high-quality cancer care delivery may coexist within Veterans Affairs hospitals. Key Words: cancer care, quality measurement, composites (Med Care 2015;53: 54–64)
T
he Institute of Medicine report, Delivering High-Quality Cancer Care: Charting a New Course for a System in Crisis, identified several components of a high-quality cancer care delivery system and recommended development of a national quality reporting program for cancer care.1 However, existing cancer care quality indicators may not be sufficient to capture the complexities and salient dimensions of cancer care quality.1 In addition, growth in the number of reported quality indicators can create challenges for users of quality data interested in summarizing quality performance and prioritizing areas for improvement. For example, the American Society of Clinical Oncology’s Quality Oncology Practice Initiative includes over 100 cancer-related quality measures,2 whereas the National Quality Measures Clearinghouse maintains a repository of nearly 200 measures spanning the cancer care continuum3; however, neither measure set summarizes quality performance along specific dimensions of cancer care. Composite measures summarize and reduce the amount of data presented in quality reports4; however, use of composite measures for cancer-related quality reporting has been limited. Outside of oncology, quality measures are commonly grouped into composites by condition (eg, heart disease, diabetes) or care modality (eg, surgery) and combined using a variety of techniques.5–9 An alternative composite method, exploratory factor analysis, identifies groups of highly correlated measures that might reflect an underlying construct. Empirically derived composite measures may better represent the underlying dimensions of care and may be more strongly linked to patient outcomes than disease-specific composite measures.10 Compared with care for other conditions, cancer care is typically more complex, including acute, chronic, and endof-life care that varies by cancer type and stage and must be coordinated across providers from multiple disciplines, including surgery, medical oncology, and radiation oncology. Given the complexity of cancer care, it is unclear which of Medical Care
Volume 53, Number 1, January 2015
Medical Care
Volume 53, Number 1, January 2015
the above-mentioned composite measure approaches is most appropriate for summarizing cancer care quality. This study extends previous work on cancer care and quality measurement by comparing empirical-factor, cancerspecific, and care modality–specific composite measures. We examine associations among the generated set of cancer care composite measures and assess how well each set of composite measures predicts survival.
METHODS Data Patient-level Data We obtained data on patients who were diagnosed with cancer and/or received their first course of cancer therapy at a Veterans Affairs (VA) Medical Center during 2001–2004 from the VA Central Cancer Registry. The VA Central Cancer Registry maintains information on patient demographics, tumor characteristics, and primary treatment for each incident cancer. Registry data were linked with additional data from 2000 to 2005, including VA administrative data (inpatient, outpatient, pharmacy, and laboratory data), Medicare administrative data (for Medicare-eligible veterans), 2000 census data to obtain zip code-level measures of socioeconomic status, and the National Death Index to determine vital status through 2005. We identified veterans with colorectal, lung, or prostate cancers—the most prevalent cancers among veterans. As described previously,11 we excluded patients whose cancers were reported based on autopsy or death certificate (N = 16 colorectal; N = 59 lung; N = 114 prostate) or for whom no reporting source was available; patients for whom data were incomplete (missing month of diagnosis, no administrative data between 45 d before diagnosis through 195 d after diagnosis), or patients with histologic features suggesting a primary cancer other than the cancer of interest.
Hospital Exclusions To ensure that we obtained reliable estimates of cancer care quality at each hospital, we excluded hospitals with no cancer patients or low cancer patient volume. Of the 128 VA Medical Centers, 10 had no cancer patients. We ranked the remaining 118 hospitals according to cancer patient volume. Appendix Table 1 (Supplemental Digital Content 1, http:// links.lww.com/MLR/A817) describes the range of eligible patients within hospitals, per quality indicator. Hospitals with overall cancer patient volumes below the median (N = 170 cancer patients) accounted for relatively few or no patients across most quality measures and were excluded in composite measure calculations, leaving 59 hospitals caring for roughly 70% of VA cancer patients.
Cancer Care Quality Measures Consistent with other literature on cancer care quality measurement,1–3,12 we defined cancer care quality as the delivery of evidence-based cancer care recommended by guidelines. We focused specifically on cancer treatment (care processes) for our 3 cancer types and colorectal cancer stage at diagnosis (a proxy for screening). We excluded structural r
2014 Lippincott Williams & Wilkins
Composite Measures of Cancer Care
and outcome measures because we were primarily interested in summarizing quality based on receipt of recommended care; also, we observed little variation in hospital structural characteristics (Appendix Table 2, Supplemental Digital Content 2, http://links.lww.com/MLR/A818). In total, we assessed 13 cancer-related process measures11,13 recommended by national evidence-based guidelines for colorectal, lung, and prostate cancer care during the study period (Table 1).14–25 We computed unadjusted hospital-level rates of recommended cancer care by aggregating VA patientlevel process measure data for each hospital.
Interunit Reliability (IUR) Because cancer patient volume varied across hospitals, we computed the average IUR for each measure, based on the average number of patients eligible for each measure, to determine how reliably each measure distinguished performance across the 59 hospitals.26 An IUR > 0.70 indicates a measure reliably distinguishes among units; however, even if IURs are low for some measures, combining low IUR measures that are empirically related into composites can improve measure reliability. Average IURs for our measures ranged from 0.12 to 0.98 with most measures (8 of 13) exhibiting an average IURZ0.50 (Table 1). We also considered adjuvant chemotherapy for stage III colon cancer, but excluded this measure because its average IUR (< 0.01) indicated no variation in hospital performance.
Survival Using vital status from the National Death Index, VA administrative data, and Medicare administrative data, we computed time to death from any cause. We censored patients alive as of December 31, 2005, the last date for which complete vital status data were available.
Analysis Although most quality indicators applied to unique patient subgroups, some patients were eligible for multiple measures depending on their stage and cancer type. Because we were interested in assessing performance on each cancer care measure and each quality measure in which a patient was eligible reflected an opportunity for providing recommended care, we treated each quality indicator as mutually exclusive when computing composite measures, consistent with previous studies.10,27,28
Empirical-Factor Composite Measures To assess correlations across hospitals among rates for the different measures, we estimated a multivariate multilevel logistic regression model for all 118 hospitals. For this particular analysis, we leveraged patient-level quality data from all 118 hospitals, instead of the 59 hospitals with higher cancer patient volumes, to obtain efficient estimates of the correlations among our quality measures. In this model, outcomes were patient-level process indicators for each measure and the only predictors were correlated hospital random effects for each measure. We estimated this model and performed a factor analysis on the estimated covariance matrix of the random effects with an oblique (Promax) www.lww-medicalcare.com |
55
56 | www.lww-medicalcare.com
r
Prostate cancer Androgen ablation within 120 d for men with stage IV prostate cancer20,22,23,25
Receipt of androgen deprivation therapy with a gonadotropin-releasing hormone
Receipt of cisplatin or carboplatin and etoposide with concurrent radiation therapy within 180 d of diagnosis; chemotherapy must start between the start and end dates of radiation therapy
Chemotherapy and radiation for limitedstage small cell lung cancer18
5009
1495
648
All stage I/II/III colon cancer patients. Patients had to be alive and not enrolled in a Medicare HMO through 180 d from surgery All stage I/II/III rectal cancer patients. Patients had to be alive and not enrolled in a Medicare HMO through 180 d from surgery All stage II/III rectal cancer patients who underwent curative-intent resection. Patients had to be alive and not enrolled in a Medicare HMO through 180 d from surgery
All prostate cancer patients with stage IV cancer at diagnosis. Patients had to be
1281
854
311
1833
3564
2047
All patients with stage I–IV rectal cancer
All stage I/II non–small cell lung cancer patients. Patients had to be alive and not enrolled in a Medicare HMO through 180 d from diagnosis. Patients were also included if they died within 180 d but underwent surgery All stage I/II non–small cell lung cancer patients who underwent lobectomy or pneumonectomy. Patients had to be alive and not enrolled in a Medicare HMO through 180 d from surgery All stage IIIA non–small cell lung cancer patients who underwent lobectomy or pneumonectomy or wedge resection. Patients had to be alive and not enrolled in a Medicare HMO through 90 d from surgery All limited-stage small cell lung cancer patients. Patients had to be alive through 45 d from diagnosis and not enrolled in a Medicare HMO through 180 d from diagnosis
7316
All patients with stage I–IV colon cancer
Cohort
74.3 (18.4)
59.7 (21.3)
69.6 (29.6)
88.5 (10.6)
54.5 (19.2)
76.4 (17.3)
75.8 (9.9)
92.9 (4.7)
58.1 (9.8)
55.3 (7.9)
0.73 [0.34–0.90]
0.40 [0.04–0.58]
0.32 [0.08–0.59]
0.68 [0.06–0.88]
0.87 [0.54–0.95]
0.32 [0.11–0.52]
0.29 [0.10–0.49]
0.72 [0.57–0.84]
0.12 [0.03–0.27]
0.62 [0.40–0.77]
Average IUR [IUR (Range)]
Receipt of chemotherapy and/or radiation therapy from 30 d before diagnosis through 90 d after date of surgery
Receipt of mediastinal evaluation from 45 d before diagnosis through the date of surgery
Receipt of pneumonectomy, lobectomy, or wedge or segmental resection within 180 d of diagnosis
Diagnoses with stage I and II vs. stage III and IV colon cancer Diagnoses with stage I and II vs. stage III and IV rectal cancer Receipt of curative resection within 180 d of diagnosis; polypectomy/local excision of the tumor for stage 1 T1 tumors that have well-differentiated or moderately differentiated tumor grades were also included Receipt of curative resection within 180 d of diagnosis; polypectomy/local excision of the tumor for stage 1 T1 tumors that have well-differentiated or moderately differentiated tumor grades were also included Receipt of both adjuvant chemotherapy with 5-fluorouracil or capecitabine and radiation therapy before or within 140 d following curative-intent resection for stage II or III rectal cancer
Definition
Mean Unadjusted Hospital Rates [% (SD)]
Medical Care
Chemotherapy and/or radiation for resected stage IIIA non–small cell lung cancer17
Mediastinal evaluation for stage I or II non–small cell lung cancer17
Lung Cancer Curative surgery for stage I or II non–small cell lung cancer17
Adjuvant chemotherapy and radiation therapy for stage II or III rectal cancer16
Curative surgery for stage I, II, or III rectal cancer16
Colorectal cancer Early stage (stage I/II vs. III/IV) at presentation, colon cancer*14 Early stage (stage I/II vs. III/IV) at presentation, rectal cancer*14 Curative surgery for stage I, II, or III colon cancer15
Quality Measures
Eligible Patients (N)
TABLE 1. Hospital-level Cancer Care Process Quality Measures, Cohort Eligibility, Eligible Patients, Mean Unadjusted Hospital-level Measure Rates, and Average Interunit Reliabilities (IURs) for 59 Veterans Affairs (VA) Hospitals
Samuel et al Volume 53, Number 1, January 2015
2014 Lippincott Williams & Wilkins
r
2014 Lippincott Williams & Wilkins 6413
3190
58.6 (28.5)
56.5 (18.1)
81.5 (13.5)
0.98 [0.81–0.99]
0.86 [0.26–0.95]
0.56 [0.16–0.81]
Cohort restrictions requiring that “patients had to be alive and not enrolled in a Medicare HMO through 90/120/180 days” were employed for measures that involved use of Medicare claims data to capture care outside of the VA. This cohort restriction was not applied to measures that were ascertained using registry or pharmacy data. *Early-stage diagnosis measures for colon and rectal cancers are intended to reflect better colorectal cancer screening practices at hospitals.
Receipt of 3D-CRT or IMRT among men with local or regional prostate cancer who received external-beam radiation therapy within 180 d of diagnosis
All prostate cancer patients with high-risk (Gleason 8–10 or PSA > 20 or stage T3 or greater), nonmetastatic tumors treated with radiation therapy within 180 d of diagnosis. Patients were required to be alive and not enrolled in a Medicare HMO through 180 d from diagnosis. We only included cases in 2001–2002 because Gleason 7 tumors could not be distinguished from Gleason 8 tumors in 2003–2004 All prostate cancer patients with local or regional prostate cancer at diagnosis who also had evidence of external-beam radiation therapy in administrative data. Patients had to be alive and not enrolled in a Medicare HMO through 180 d from diagnosis
Receipt of hormonal therapy (adjuvant or neoadjuvant)
1192
3-dimensional conformal radiotherapy (3-D CRT) or intensity-modulated radiation therapy (IMRT) for prostate cancer patients treated with electron beam radiation therapy (XRT)19,21,24
Oral antiandrogen before initiating gonadotropin-releasing hormone (GnRH) agonist therapy for metastatic prostate cancer19 Adjuvant androgen deprivation therapy for high-risk cancers treated with radiation therapy19
alive and not enrolled in a Medicare HMO through 120 d from diagnosis All prostate cancer patients with stage IV metastatic cancer at diagnosis who started a GnRH agonist
(GnRH) agonist or bilateral orchiectomy within 120 d of diagnosis A filled prescription for an oral antiandrogen for at least 2 wk, beginning at least 1 wk before first dose of GnRH agonist
Medical Care Volume 53, Number 1, January 2015 Composite Measures of Cancer Care
www.lww-medicalcare.com |
57
Medical Care
Samuel et al
Volume 53, Number 1, January 2015
TABLE 2. Patient Characteristics Across Cancer Cohorts Colon
Rectal
Non–Small Cell Lung
Small Cell Lung
Prostate
N = 7725
N = 2233
N = 17,511
N = 2875
N = 31,238
28.4 12.7 13.0 45.9
23.6 14.1 15.9 46.4
25.0 16.6 15.4 43.0
22.4 16.3 20.2 41.1
76.4 14.4 6.7 2.5
76.1 20.1 2.1 1.7
82.3 14.0 2.2 1.6
62.2 27.2 7.0 3.7
1.0 99.0
1.7 98.3
2.0 98.0
100.0
40.2 49.3 10.6
53.0 45.5 1.6
51.1 47.8 1.1
45.1 53.3 1.7
87.1 13.0
82.0 18.0
84.4 15.7
93.1 7.0
61.2 24.3 8.3 6.1
61.5 22.3 9.5 6.7
61.6 21.6 9.6 7.3
63.3 24.5 7.7 4.5
23.1 24.3 25.0 27.6
24.7 24.8 25.3 25.2
23.5 24.8 25.0 26.7
24.5 25.3 25.1 25.1
34.5 24.0 18.9 16.2 6.5
25.4 7.1 26.7 37.7 3.2
60.2w 3.7
24.4 22.6 23.1 25.3 4.7
23.8 22.0 23.6 25.8 4.8
25.3 24.1 22.7 23.9 5.0
23.7 21.4 23.0 26.8 5.1
25.9 22.4 23.3 23.7 4.6 50.9
24.3 23.3 22.9 24.7 4.8 58.0
24.4 23.7 22.4 24.5 5.0 56.5
24.3 22.8 23.0 24.8 5.1 49.6
Characteristics (%) Age (y) < 60 20.3 60–64 12.4 65–69 14.0 Z70 53.3 Race White 71.3 Black 19.8 Hispanic 6.5 Other 2.5 Sex Female 2.0 Male 98.0 Marital status Unmarried 39.4 Married 50.7 Unknown 9.9 Prior history of cancer No 85.8 Yes 14.2 Charlson Comorbidity Score 0 51.2 1 28.3 2 11.7 3+ 8.8 Year of diagnosis 2001 24.1 2002 25.6 2003 25.3 2004 25.0 Tumor stage Stage I (least advanced) 30.0 Stage II 25.3 Stage III 21.6 Stage IV (most advanced) 18.4 Stage missing/unknown 4.7 % Population Z65 y living below poverty in zip code of residence Q1 (0%– < 7.9%) 23.8 Q2 (7.9%– < 12.8%) 21.8 Q3 (12.8%– < 19.5%) 23.0 Q4 (19.5%–76.9%) 26.1 Missing/unknown 5.4 % Population college graduates in zip code of residence Q1 (< 15.9%) 23.6 Q2 (15.9%– < 21.6%) 22.7 Q3 (21.6%– < 30.0%) 22.6 Q4 (30.0%–100.0%) 25.8 Missing/unknown 5.4 Average distance to reporting hospital (miles) 50.6
36.1w
81.8* 7.8* 4.9* 5.6
*Prostate cancer stage categories (in order from least advanced to most advanced): local, regional, distant. w Small cell lung cancer stage categories (in order from least advanced to most advanced): limited stage, extensive stage.
rotation, which allowed for correlations among factors,28 using the “glmer” and “factanal” functions, respectively, in R V.3.0.2 (Vienna, Austria). We assigned measures to the factor domain on which its loading was largest (dominant factor loading), ignoring absolute loadings
Developing and Evaluating Composite Measures of Cancer Care Quality Cleo A. Samuel, PhD,* Alan M. Zaslavsky, PhD,w Mary Beth Landrum, PhD,w Karl Lorenz, MD, MSHS,zy and Nancy L. Keating, MD, MPHw8
Background: Composite measures are useful for distilling quality data into summary scores; yet, there has been limited use of composite measures for cancer care. Objective: Compare multiple approaches for generating cancer care composite measures and evaluate how well composite measures summarize dimensions of cancer care and predict survival. Study Design: We computed hospital-level rates for 13 colorectal, lung, and prostate cancer process measures in 59 Veterans Affairs hospitals. We computed 4 empirical-factor (based on an exploratory factor analysis), 3 cancer-specific (colorectal, lung, prostate care), and 3 care modality–specific (diagnosis/evaluation, surgical, nonsurgical treatments) composite measures. We assessed correlations among all composite measures and estimated all-cause survival for colon, rectal, non–small cell lung, and small cell lung cancers as a function of composite scores, adjusting for patient characteristics. Results: Four factors emerged from the factor analysis: nonsurgical treatment, surgical treatment, colorectal early diagnosis, and prostate treatment. We observed strong correlations (r) among composite measures comprised of similar process measures (r = 0.58–1.00, P < 0.0001), but not among composite measures reflecting different care dimensions. Composite measures were rarely associated with survival. Conclusions: The empirical-factor domains grouped measures variously by cancer type and care modality. The evidence did not
From the *Department of Health Policy and Management, Gillings School of Global Public Health, University of North Carolina at Chapel Hill, Chapel Hill, NC; wDepartment of Health Care Policy, Harvard Medical School, Boston, MA; zDivisions of General Internal Medicine and Palliative Care, Veterans Administration Greater Los Angeles Healthcare System; yGeffen School of Medicine, University of California Los Angeles, Los Angeles, CA; and 8Division of General Internal Medicine, Brigham and Women’s Hospital, Boston, MA. Supported by the Department of Veterans Affairs through the Office of Policy and Planning as part of an evaluation of oncology care. C.A.S.’s effort was supported by the Ruth L. Kirschstein National Research Service Award and the Harvard University Graduate Prize Fellowship. The authors declare no conflict of interest. Reprints: Nancy L. Keating, MD, MPH, Department of Health Care Policy, Harvard Medical School, 180 Longwood Avenue, Boston, MA 02115. E-mail: [email protected]. Supplemental Digital Content is available for this article. Direct URL citations appear in the printed text and are provided in the HTML and PDF versions of this article on the journal’s Website, www.lwwmedicalcare.com. Copyright r 2014 by Lippincott Williams & Wilkins ISSN: 0025-7079/15/5301-0054
54 | www.lww-medicalcare.com
support any single approach for generating cancer care composite measures. Weak associations across different care domains suggest that low-quality and high-quality cancer care delivery may coexist within Veterans Affairs hospitals. Key Words: cancer care, quality measurement, composites (Med Care 2015;53: 54–64)
T
he Institute of Medicine report, Delivering High-Quality Cancer Care: Charting a New Course for a System in Crisis, identified several components of a high-quality cancer care delivery system and recommended development of a national quality reporting program for cancer care.1 However, existing cancer care quality indicators may not be sufficient to capture the complexities and salient dimensions of cancer care quality.1 In addition, growth in the number of reported quality indicators can create challenges for users of quality data interested in summarizing quality performance and prioritizing areas for improvement. For example, the American Society of Clinical Oncology’s Quality Oncology Practice Initiative includes over 100 cancer-related quality measures,2 whereas the National Quality Measures Clearinghouse maintains a repository of nearly 200 measures spanning the cancer care continuum3; however, neither measure set summarizes quality performance along specific dimensions of cancer care. Composite measures summarize and reduce the amount of data presented in quality reports4; however, use of composite measures for cancer-related quality reporting has been limited. Outside of oncology, quality measures are commonly grouped into composites by condition (eg, heart disease, diabetes) or care modality (eg, surgery) and combined using a variety of techniques.5–9 An alternative composite method, exploratory factor analysis, identifies groups of highly correlated measures that might reflect an underlying construct. Empirically derived composite measures may better represent the underlying dimensions of care and may be more strongly linked to patient outcomes than disease-specific composite measures.10 Compared with care for other conditions, cancer care is typically more complex, including acute, chronic, and endof-life care that varies by cancer type and stage and must be coordinated across providers from multiple disciplines, including surgery, medical oncology, and radiation oncology. Given the complexity of cancer care, it is unclear which of Medical Care
Volume 53, Number 1, January 2015
Medical Care
Volume 53, Number 1, January 2015
the above-mentioned composite measure approaches is most appropriate for summarizing cancer care quality. This study extends previous work on cancer care and quality measurement by comparing empirical-factor, cancerspecific, and care modality–specific composite measures. We examine associations among the generated set of cancer care composite measures and assess how well each set of composite measures predicts survival.
METHODS Data Patient-level Data We obtained data on patients who were diagnosed with cancer and/or received their first course of cancer therapy at a Veterans Affairs (VA) Medical Center during 2001–2004 from the VA Central Cancer Registry. The VA Central Cancer Registry maintains information on patient demographics, tumor characteristics, and primary treatment for each incident cancer. Registry data were linked with additional data from 2000 to 2005, including VA administrative data (inpatient, outpatient, pharmacy, and laboratory data), Medicare administrative data (for Medicare-eligible veterans), 2000 census data to obtain zip code-level measures of socioeconomic status, and the National Death Index to determine vital status through 2005. We identified veterans with colorectal, lung, or prostate cancers—the most prevalent cancers among veterans. As described previously,11 we excluded patients whose cancers were reported based on autopsy or death certificate (N = 16 colorectal; N = 59 lung; N = 114 prostate) or for whom no reporting source was available; patients for whom data were incomplete (missing month of diagnosis, no administrative data between 45 d before diagnosis through 195 d after diagnosis), or patients with histologic features suggesting a primary cancer other than the cancer of interest.
Hospital Exclusions To ensure that we obtained reliable estimates of cancer care quality at each hospital, we excluded hospitals with no cancer patients or low cancer patient volume. Of the 128 VA Medical Centers, 10 had no cancer patients. We ranked the remaining 118 hospitals according to cancer patient volume. Appendix Table 1 (Supplemental Digital Content 1, http:// links.lww.com/MLR/A817) describes the range of eligible patients within hospitals, per quality indicator. Hospitals with overall cancer patient volumes below the median (N = 170 cancer patients) accounted for relatively few or no patients across most quality measures and were excluded in composite measure calculations, leaving 59 hospitals caring for roughly 70% of VA cancer patients.
Cancer Care Quality Measures Consistent with other literature on cancer care quality measurement,1–3,12 we defined cancer care quality as the delivery of evidence-based cancer care recommended by guidelines. We focused specifically on cancer treatment (care processes) for our 3 cancer types and colorectal cancer stage at diagnosis (a proxy for screening). We excluded structural r
2014 Lippincott Williams & Wilkins
Composite Measures of Cancer Care
and outcome measures because we were primarily interested in summarizing quality based on receipt of recommended care; also, we observed little variation in hospital structural characteristics (Appendix Table 2, Supplemental Digital Content 2, http://links.lww.com/MLR/A818). In total, we assessed 13 cancer-related process measures11,13 recommended by national evidence-based guidelines for colorectal, lung, and prostate cancer care during the study period (Table 1).14–25 We computed unadjusted hospital-level rates of recommended cancer care by aggregating VA patientlevel process measure data for each hospital.
Interunit Reliability (IUR) Because cancer patient volume varied across hospitals, we computed the average IUR for each measure, based on the average number of patients eligible for each measure, to determine how reliably each measure distinguished performance across the 59 hospitals.26 An IUR > 0.70 indicates a measure reliably distinguishes among units; however, even if IURs are low for some measures, combining low IUR measures that are empirically related into composites can improve measure reliability. Average IURs for our measures ranged from 0.12 to 0.98 with most measures (8 of 13) exhibiting an average IURZ0.50 (Table 1). We also considered adjuvant chemotherapy for stage III colon cancer, but excluded this measure because its average IUR (< 0.01) indicated no variation in hospital performance.
Survival Using vital status from the National Death Index, VA administrative data, and Medicare administrative data, we computed time to death from any cause. We censored patients alive as of December 31, 2005, the last date for which complete vital status data were available.
Analysis Although most quality indicators applied to unique patient subgroups, some patients were eligible for multiple measures depending on their stage and cancer type. Because we were interested in assessing performance on each cancer care measure and each quality measure in which a patient was eligible reflected an opportunity for providing recommended care, we treated each quality indicator as mutually exclusive when computing composite measures, consistent with previous studies.10,27,28
Empirical-Factor Composite Measures To assess correlations across hospitals among rates for the different measures, we estimated a multivariate multilevel logistic regression model for all 118 hospitals. For this particular analysis, we leveraged patient-level quality data from all 118 hospitals, instead of the 59 hospitals with higher cancer patient volumes, to obtain efficient estimates of the correlations among our quality measures. In this model, outcomes were patient-level process indicators for each measure and the only predictors were correlated hospital random effects for each measure. We estimated this model and performed a factor analysis on the estimated covariance matrix of the random effects with an oblique (Promax) www.lww-medicalcare.com |
55
56 | www.lww-medicalcare.com
r
Prostate cancer Androgen ablation within 120 d for men with stage IV prostate cancer20,22,23,25
Receipt of androgen deprivation therapy with a gonadotropin-releasing hormone
Receipt of cisplatin or carboplatin and etoposide with concurrent radiation therapy within 180 d of diagnosis; chemotherapy must start between the start and end dates of radiation therapy
Chemotherapy and radiation for limitedstage small cell lung cancer18
5009
1495
648
All stage I/II/III colon cancer patients. Patients had to be alive and not enrolled in a Medicare HMO through 180 d from surgery All stage I/II/III rectal cancer patients. Patients had to be alive and not enrolled in a Medicare HMO through 180 d from surgery All stage II/III rectal cancer patients who underwent curative-intent resection. Patients had to be alive and not enrolled in a Medicare HMO through 180 d from surgery
All prostate cancer patients with stage IV cancer at diagnosis. Patients had to be
1281
854
311
1833
3564
2047
All patients with stage I–IV rectal cancer
All stage I/II non–small cell lung cancer patients. Patients had to be alive and not enrolled in a Medicare HMO through 180 d from diagnosis. Patients were also included if they died within 180 d but underwent surgery All stage I/II non–small cell lung cancer patients who underwent lobectomy or pneumonectomy. Patients had to be alive and not enrolled in a Medicare HMO through 180 d from surgery All stage IIIA non–small cell lung cancer patients who underwent lobectomy or pneumonectomy or wedge resection. Patients had to be alive and not enrolled in a Medicare HMO through 90 d from surgery All limited-stage small cell lung cancer patients. Patients had to be alive through 45 d from diagnosis and not enrolled in a Medicare HMO through 180 d from diagnosis
7316
All patients with stage I–IV colon cancer
Cohort
74.3 (18.4)
59.7 (21.3)
69.6 (29.6)
88.5 (10.6)
54.5 (19.2)
76.4 (17.3)
75.8 (9.9)
92.9 (4.7)
58.1 (9.8)
55.3 (7.9)
0.73 [0.34–0.90]
0.40 [0.04–0.58]
0.32 [0.08–0.59]
0.68 [0.06–0.88]
0.87 [0.54–0.95]
0.32 [0.11–0.52]
0.29 [0.10–0.49]
0.72 [0.57–0.84]
0.12 [0.03–0.27]
0.62 [0.40–0.77]
Average IUR [IUR (Range)]
Receipt of chemotherapy and/or radiation therapy from 30 d before diagnosis through 90 d after date of surgery
Receipt of mediastinal evaluation from 45 d before diagnosis through the date of surgery
Receipt of pneumonectomy, lobectomy, or wedge or segmental resection within 180 d of diagnosis
Diagnoses with stage I and II vs. stage III and IV colon cancer Diagnoses with stage I and II vs. stage III and IV rectal cancer Receipt of curative resection within 180 d of diagnosis; polypectomy/local excision of the tumor for stage 1 T1 tumors that have well-differentiated or moderately differentiated tumor grades were also included Receipt of curative resection within 180 d of diagnosis; polypectomy/local excision of the tumor for stage 1 T1 tumors that have well-differentiated or moderately differentiated tumor grades were also included Receipt of both adjuvant chemotherapy with 5-fluorouracil or capecitabine and radiation therapy before or within 140 d following curative-intent resection for stage II or III rectal cancer
Definition
Mean Unadjusted Hospital Rates [% (SD)]
Medical Care
Chemotherapy and/or radiation for resected stage IIIA non–small cell lung cancer17
Mediastinal evaluation for stage I or II non–small cell lung cancer17
Lung Cancer Curative surgery for stage I or II non–small cell lung cancer17
Adjuvant chemotherapy and radiation therapy for stage II or III rectal cancer16
Curative surgery for stage I, II, or III rectal cancer16
Colorectal cancer Early stage (stage I/II vs. III/IV) at presentation, colon cancer*14 Early stage (stage I/II vs. III/IV) at presentation, rectal cancer*14 Curative surgery for stage I, II, or III colon cancer15
Quality Measures
Eligible Patients (N)
TABLE 1. Hospital-level Cancer Care Process Quality Measures, Cohort Eligibility, Eligible Patients, Mean Unadjusted Hospital-level Measure Rates, and Average Interunit Reliabilities (IURs) for 59 Veterans Affairs (VA) Hospitals
Samuel et al Volume 53, Number 1, January 2015
2014 Lippincott Williams & Wilkins
r
2014 Lippincott Williams & Wilkins 6413
3190
58.6 (28.5)
56.5 (18.1)
81.5 (13.5)
0.98 [0.81–0.99]
0.86 [0.26–0.95]
0.56 [0.16–0.81]
Cohort restrictions requiring that “patients had to be alive and not enrolled in a Medicare HMO through 90/120/180 days” were employed for measures that involved use of Medicare claims data to capture care outside of the VA. This cohort restriction was not applied to measures that were ascertained using registry or pharmacy data. *Early-stage diagnosis measures for colon and rectal cancers are intended to reflect better colorectal cancer screening practices at hospitals.
Receipt of 3D-CRT or IMRT among men with local or regional prostate cancer who received external-beam radiation therapy within 180 d of diagnosis
All prostate cancer patients with high-risk (Gleason 8–10 or PSA > 20 or stage T3 or greater), nonmetastatic tumors treated with radiation therapy within 180 d of diagnosis. Patients were required to be alive and not enrolled in a Medicare HMO through 180 d from diagnosis. We only included cases in 2001–2002 because Gleason 7 tumors could not be distinguished from Gleason 8 tumors in 2003–2004 All prostate cancer patients with local or regional prostate cancer at diagnosis who also had evidence of external-beam radiation therapy in administrative data. Patients had to be alive and not enrolled in a Medicare HMO through 180 d from diagnosis
Receipt of hormonal therapy (adjuvant or neoadjuvant)
1192
3-dimensional conformal radiotherapy (3-D CRT) or intensity-modulated radiation therapy (IMRT) for prostate cancer patients treated with electron beam radiation therapy (XRT)19,21,24
Oral antiandrogen before initiating gonadotropin-releasing hormone (GnRH) agonist therapy for metastatic prostate cancer19 Adjuvant androgen deprivation therapy for high-risk cancers treated with radiation therapy19
alive and not enrolled in a Medicare HMO through 120 d from diagnosis All prostate cancer patients with stage IV metastatic cancer at diagnosis who started a GnRH agonist
(GnRH) agonist or bilateral orchiectomy within 120 d of diagnosis A filled prescription for an oral antiandrogen for at least 2 wk, beginning at least 1 wk before first dose of GnRH agonist
Medical Care Volume 53, Number 1, January 2015 Composite Measures of Cancer Care
www.lww-medicalcare.com |
57
Medical Care
Samuel et al
Volume 53, Number 1, January 2015
TABLE 2. Patient Characteristics Across Cancer Cohorts Colon
Rectal
Non–Small Cell Lung
Small Cell Lung
Prostate
N = 7725
N = 2233
N = 17,511
N = 2875
N = 31,238
28.4 12.7 13.0 45.9
23.6 14.1 15.9 46.4
25.0 16.6 15.4 43.0
22.4 16.3 20.2 41.1
76.4 14.4 6.7 2.5
76.1 20.1 2.1 1.7
82.3 14.0 2.2 1.6
62.2 27.2 7.0 3.7
1.0 99.0
1.7 98.3
2.0 98.0
100.0
40.2 49.3 10.6
53.0 45.5 1.6
51.1 47.8 1.1
45.1 53.3 1.7
87.1 13.0
82.0 18.0
84.4 15.7
93.1 7.0
61.2 24.3 8.3 6.1
61.5 22.3 9.5 6.7
61.6 21.6 9.6 7.3
63.3 24.5 7.7 4.5
23.1 24.3 25.0 27.6
24.7 24.8 25.3 25.2
23.5 24.8 25.0 26.7
24.5 25.3 25.1 25.1
34.5 24.0 18.9 16.2 6.5
25.4 7.1 26.7 37.7 3.2
60.2w 3.7
24.4 22.6 23.1 25.3 4.7
23.8 22.0 23.6 25.8 4.8
25.3 24.1 22.7 23.9 5.0
23.7 21.4 23.0 26.8 5.1
25.9 22.4 23.3 23.7 4.6 50.9
24.3 23.3 22.9 24.7 4.8 58.0
24.4 23.7 22.4 24.5 5.0 56.5
24.3 22.8 23.0 24.8 5.1 49.6
Characteristics (%) Age (y) < 60 20.3 60–64 12.4 65–69 14.0 Z70 53.3 Race White 71.3 Black 19.8 Hispanic 6.5 Other 2.5 Sex Female 2.0 Male 98.0 Marital status Unmarried 39.4 Married 50.7 Unknown 9.9 Prior history of cancer No 85.8 Yes 14.2 Charlson Comorbidity Score 0 51.2 1 28.3 2 11.7 3+ 8.8 Year of diagnosis 2001 24.1 2002 25.6 2003 25.3 2004 25.0 Tumor stage Stage I (least advanced) 30.0 Stage II 25.3 Stage III 21.6 Stage IV (most advanced) 18.4 Stage missing/unknown 4.7 % Population Z65 y living below poverty in zip code of residence Q1 (0%– < 7.9%) 23.8 Q2 (7.9%– < 12.8%) 21.8 Q3 (12.8%– < 19.5%) 23.0 Q4 (19.5%–76.9%) 26.1 Missing/unknown 5.4 % Population college graduates in zip code of residence Q1 (< 15.9%) 23.6 Q2 (15.9%– < 21.6%) 22.7 Q3 (21.6%– < 30.0%) 22.6 Q4 (30.0%–100.0%) 25.8 Missing/unknown 5.4 Average distance to reporting hospital (miles) 50.6
36.1w
81.8* 7.8* 4.9* 5.6
*Prostate cancer stage categories (in order from least advanced to most advanced): local, regional, distant. w Small cell lung cancer stage categories (in order from least advanced to most advanced): limited stage, extensive stage.
rotation, which allowed for correlations among factors,28 using the “glmer” and “factanal” functions, respectively, in R V.3.0.2 (Vienna, Austria). We assigned measures to the factor domain on which its loading was largest (dominant factor loading), ignoring absolute loadings
