Medicare Claims Versus Beneficiary Self-Report for Influenza Vaccination Surveillance Kimberly A. Lochner, ScD, Marc A. Wynne, MSPH, Gloria H. Wheatcroft, MPH, Chris M. Worrall, BS, Jeffrey A. Kelman, MD Background: Although self-reported influenza vaccination status is routinely used in surveillance to estimate influenza vaccine coverage, Medicare data are becoming a promising resource for influenza surveillance to inform vaccination program management and planning.
Purpose: To evaluate the concordance between self-reported influenza vaccination and influenza vaccination claims among Medicare beneficiaries.
Methods: This study compared influenza vaccination based upon Medicare claims and self-report among a sample of Medicare beneficiaries (N¼9,378) from the 2011 Medicare Current Beneficiary Survey, which was the most recent year of data at the time of analysis (summer 2013). Sensitivity, specificity, positive predictive value, and negative predictive value were calculated using selfreported data as the referent standard. Logistic regression was used to compute the marginal mean proportions for whether a Medicare influenza vaccination claim was present among beneficiaries who reported receiving the vaccination.
Results: Influenza vaccination was higher for self-report (69.4%) than Medicare claims (48.3%). For Medicare claims, sensitivity¼67.5%, specificity¼96.3%, positive predictive value¼97.6%, and negative predictive value¼56.7%. Among beneficiaries reporting receiving an influenza vaccination, the percentage of beneficiaries with a vaccination claim was lower for beneficiaries who were aged o65 years, male, non-Hispanic black or Hispanic, and had less than a college education. Conclusions: The classification of influenza vaccination status for Medicare beneficiaries can differ based upon survey and claims. To improve Medicare claims–based surveillance studies, further research is needed to determine the sources of discordance in self-reported and Medicare claims data, specifically for sensitivity and negative predictive value. (Am J Prev Med 2015;48(4):384–391) Published by Elsevier Inc. on behalf of American Journal of Preventive Medicine
Introduction
A
n annual influenza vaccination is recommended for all people aged Z6 months in the U.S. who do not have contraindications to the vaccine.1 Surveillance of influenza vaccination coverage is essential in determining season-specific estimates and trends as well as From the Centers for Medicare & Medicaid Services, Sam Nunn Atlanta Federal Center (Lochner), Atlanta, Georgia; Office of Information Products and Data Analytics (Lochner, Wheatcroft); and Center for Medicare (Wynne, Worrall, Kelman), Baltimore, Maryland Address correspondence to: Kimberly A. Lochner, ScD, CMS, Sam Nunn Atlanta Federal Center, 61 Forsyth Street, S.W., Suite 4T20, Atlanta GA 30303. E-mail: [email protected]. 0749-3797/$36.00 http://dx.doi.org/10.1016/j.amepre.2014.10.016
384
Am J Prev Med 2015;48(4):384–391
identifying groups with lower coverage by geographic area, age, race/ethnicity, and SES.2 Although self-reported influenza vaccination status, collected in national surveys, is routinely used in surveillance to estimate influenza vaccine coverage across the U.S. (www.cdc.gov/flu/fluvaxview/ index.htmh), Medicare data are becoming a promising resource for influenza surveillance to inform vaccination program management and planning. Medicare claims data cover a large proportion of the population aged Z65 years, a population at high risk for complications due to influenza. Medicare claims data have other advantages, such as the ability to generate small-area estimates, as well as near “realtime” seasonal information.3,4 Medicare claims have been used to monitor associations between influenza vaccinations and rare adverse
Published by Elsevier Inc. on behalf of American Journal of Preventive Medicine
Lochner et al / Am J Prev Med 2015;48(4):384–391 4–6
events and recently to track influenza vaccination coverage and disparities in coverage for the Medicare population (www.hhs.gov/nvpo/flu-vaccination-map/ index.html). Yet, compared with other data sources that assess influenza vaccination coverage, those based upon Medicare claims submitted for reimbursement for influenza vaccine are substantially lower. A study of people aged 65–79 years in Allegheny County PA found that influenza vaccination was estimated to be 75% based upon self-report and only 36% based upon Medicare claims.7 Influenza vaccination coverage based upon several surveys was 65%–70%, whereas coverage based upon Medicare claims was just more than 40%.5 Because of the substantial discrepancy between Medicare claims and survey self-report, studies of influenza surveillance may be hindered by the potential misclassification of a beneficiary’s influenza vaccination status depending upon the data source. Direct comparisons between claims and self-report among the same Medicare beneficiaries are needed to better understand the strengths and limitations of specific data sources so that the data used to assess vaccination status can be improved. This study directly compared Medicare beneficiaries’ claims and their self-reported information using a sample of respondents to the Medicare Current Beneficiary Survey (MCBS) who were linked to their Medicare enrollment and claims information. The goal of this study was to assess the concordance of Medicare claims for influenza vaccination and self-reported vaccination status, quantify the sensitivity of ascertainment of vaccination in the claims, and determine the specific populations for whom the discordance is greatest. This study also explores where beneficiaries reported receiving the influenza vaccination as a means to help identify possible reasons for the discordance.
Methods Data Source and Study Population Medicare is the U.S. federal health insurance program for individuals aged Z65 years, those aged o65 years with certain disabilities, and those of any age with end-stage renal disease (ESRD). To examine the concordance between Medicare beneficiaries’ self-report of influenza vaccination and their Medicare claims, information from two data sources available from the Centers for Medicare & Medicaid Services (CMS) were linked. The first source was the MCBS, a continuous, multipurpose survey of a nationally representative sample of elderly, disabled, and institutionalized Medicare beneficiaries. The MCBS collects information on sociodemographic characteristics and health status of Medicare beneficiaries, as well as their use of and satisfaction with their health care (www.resdac.org/cms-data/files/mcbs-access-care). Information on influenza vaccination came from the 2011 MCBS April 2015
385
Access to Care file fall round interviews, which was the most recent year of data at the time of analysis (summer 2013). The second source was Medicare claims data available from the Chronic Conditions Warehouse (CCW), a research database with 100% of Medicare enrollment and fee-for-service institutional and non-institutional claims data (www.ccwdata.org). The use of these secondary, existing data sources was exempt from IRB approval as the participants in the MCBS provided informed consent; the data were de-identified and pose no risk to loss of confidentiality. The study population included respondents who were community-dwelling, survived the entire year (2011), responded yes or no to the question asking whether they had received an influenza vaccine, and had non-missing values for the sociodemographic variables of interest. MCBS respondents meeting these criteria were linked to the Medicare administrative data so that their claims could be searched for influenza vaccination codes for the period from September to December 2010, as this is the reference period for the influenza question on the MCBS. Because claims data are not available for beneficiaries enrolled in Medicare Advantage, the linked MCBS respondents had to be continuously enrolled in fee-for-service Medicare parts A and B for the period September to December 2010, which was determined from the Medicare enrollment data. This resulted in a study population of 9,378, representing approximately 27.8 million fee-for-service Medicare beneficiaries. This linkage at the beneficiary level allowed for direct comparisons between beneficiaries’ self-reported influenza vaccination and vaccination documented by the presence of Medicare claims.
Influenza Measures The 2011 MCBS Access to Care file, which was fielded during fall 2011, asked beneficiaries if they had received a seasonal influenza vaccine for the prior year, during the period September to December 2010. Medicare beneficiaries’ self-reported (yes or no) influenza vaccination status was based upon this response. For these same beneficiaries, their institutional and non-institutional claims for September to December 2010 were examined. An influenza vaccination was identified in the Medicare claims by the following Healthcare Common Procedure Coding System (HCPCS) codes: 90655–90658, 90660, 90662, Q2035–Q2039, and G0008. In addition, to address the availability of influenza vaccinations before September and after December, and given that beneficiaries may not accurately recall when they received the vaccination, a secondary analysis broadened the period of claims searched to include August 2010 to May 2011. Only 3.5% (n¼165) of beneficiaries in the linked study sample with an influenza vaccination claim had a claim outside September to December 2010, which did not affect the results.
Medicare Beneficiary Characteristics and SelfReported Place of Service for Influenza Vaccination The study examined standard sociodemographic characteristics of the study population available from the MCBS: age (categorized as o65 years and Z65 years); sex; marital status (married/widowed or divorced/separated/never married); education (less than a college degree and college degree or more); metropolitan residence (versus not); dual eligibility for Medicaid benefits (versus not); and race/ ethnicity (non-Hispanic white, non-Hispanic black or African American, Hispanic, Asian or Pacific Islander, and all other races).
386
Lochner et al / Am J Prev Med 2015;48(4):384–391
For those MCBS respondents who reported that they received an influenza vaccination, a follow-up question asked where they went for their vaccine. These responses were coded into 21 categories, which were then collapsed into five categories: (1) physician/doctor office; (2) community setting (e.g., health fair, shopping mall/store/pharmacy, senior center, church, school); (3) facility/ambulatory treatment center (e.g., freestanding surgical centers, urgent care centers, emergency department, Veterans Affairs [VA] facility); (4) clinics (e.g., rural clinics, neighborhood health centers) or health department; and (5) all others, which, because of small sample sizes, includes place of work or employment, managed care plan center or HMO, or unknown. Also owing to small sample sizes, this information is presented only for non-Hispanic whites, non-Hispanic blacks, and Hispanics.
Statistical Analysis First, the study examined the concordance between Medicare claims with influenza vaccination HCPCS or procedure codes and selfreported influenza vaccination in the sample of MCBS respondents linked to their claims data. As several studies have demonstrated selfreported influenza vaccination to be a highly sensitive (Z95%) and variably specific (38%–90%) indicator of actual vaccine status compared to medical records,8–12 the MCBS self-reported information was used as the referent standard. Overall agreement between the two data sources as well as the sensitivity; specificity; positive predictive value (PPV); and negative predictive value (NPV) of the Medicare claims data was examined. Overall agreement was calculated as the percentage of true positives and true negatives, meaning agreement from both data sources, out of the entire study population. Sensitivity refers to the probability of the presence of an influenza vaccination claim if there is self-reported influenza vaccination; specificity refers to the probability of the absence of an influenza vaccination claim if there is no self-reported influenza vaccination. PPV refers to the probability that there is self-reported influenza vaccination given that an influenza vaccination claim is present, and NPV refers to the probability that there is no self-reported influenza vaccination given that there is no influenza vaccination claim present. The study also examined factors associated with a beneficiary having a Medicare claim for influenza vaccination among beneficiaries who reported receiving the vaccination. Using a logistic regression model that included the beneficiary characteristics described above as the predictors, the marginal mean proportions were computed for whether a Medicare influenza vaccination claim was present among beneficiaries who reported receiving the vaccination. Finally, the study described where beneficiaries reported they received the influenza vaccination, using the place of service categories described above, and whether a Medicare claim was present. To take into account the complex survey design of the MCBS, all analyses were conducted using SAS, version 9.3 SURVEY procedures. Sample weights, which account for the differential probabilities of selection, non-response, and non-coverage, were incorporated into the estimation process. The SEs of the percentages were estimated using balanced repeated replicates (BRR), a method that incorporates the sample weights and sample design.13
Results The study population was composed mainly of Medicare beneficiaries who were aged Z65 years (82.3%); women
(55.2%); non-Hispanic whites (79.2%); and those living in a metropolitan area (71.8%). Approximately one fifth had at least a college degree (21.7%), and 17.2% were eligible for Medicaid benefits (“dual eligibles”; data not shown). Table 1 shows the percentage of Medicare beneficiaries receiving an influenza vaccination based upon source (self-report versus Medicare claims). Influenza vaccination coverage was 69.4% based upon selfreport and 48.3% based upon Medicare claims. Coverage was lower based upon claims data compared to survey data for all groups, with the largest differences (425 percentage points) observed for beneficiaries aged o65 years of age; non-Hispanic blacks, Hispanics, or Asian/ Pacific Islanders; and those also enrolled in Medicaid (dual-eligibles). Table 2 presents the results of the concordance between the two data sources overall and by sociodemographic characteristics. Overall, the agreement between the two data sources was 76.3% and ranged from 79.7% (for beneficiaries with at least a college degree) to 67.2% (for Hispanic beneficiaries). The sensitivity of the Medicare claims was 67.5% and the specificity was 96.3%; PPV was 97.6% and NPV was 56.6%. Sensitivity was 50.6% for beneficiaries aged o65 years and 70.2% for those aged Z65 years; it was higher for women (70.8%) than men (63.2%), highest for non-Hispanic whites (70.9%), and lowest for non-Hispanic blacks (46.9%). Sensitivity also was lower for those with less than a college degree (65.3%) and dual-eligible beneficiaries (55.1%). Specificity and PPV were high for all groups (at least 94%), whereas NPV was lower than 60% for most groups and lowest for Asian/Pacific Islanders (47.1%). Table 3 presents the predicted marginal mean proportions of having a Medicare influenza vaccination claim, controlling for all variables examined in the logistic model, among Medicare beneficiaries reporting they received the influenza vaccination. Age, sex, marital status, race/ethnicity, education level, and urban/rural residence were statistically significant (po0.01) predictors of whether a Medicare influenza vaccination claim was present. The percentage of beneficiaries with a claim was lower for beneficiaries who were aged o65 years, male, and non-Hispanic black or Hispanic and had less than a college education. The lowest percentage was among non-Hispanic blacks, where 43.6% who reported receiving the vaccination had a claim. Table 4 presents descriptive findings examining where beneficiaries reported they received the influenza vaccination and whether a Medicare claim was present. Among beneficiaries who reported receiving an influenza vaccination, more than half (53.9%) reported receiving it in a physician’s office, varying from 53% to 65% www.ajpmonline.org
Lochner et al / Am J Prev Med 2015;48(4):384–391 a
Table 1. Influenza Vaccination Coverage: MCBS-Linked Sample, 2011 Self-report (n [%]b)
Medicare claims (n [%]b)
Difference: selfreport – claims
6,508 (69.4)
4,494 (48.3)
21.1
Less than 65
955 (54.5)
497 (29.2)
25.3
65 and older
5,553 (72.6)
3,997 (52.0)
20.6
Men
2,868 (67.8)
1,874 (44.1)
23.7
Women
3,640 (70.8)
2,620 (51.2)
Married or widowed
5,163 (72.6)
3,693 (51.7)
Divorced, separated, never married
1,345 (59.2)
801 (35.9)
Non-Hispanic white
5,304 (71.6)
3,846 (51.8)
Non-Hispanic black
514 (54.9)
255 (27.3)
Asian/Pacific Islander
106 (72.4)
68 (43.0)
Hispanic
384 (63.7)
205 (35.4)
Other
200 (66.2)
120 (42.0)
Less than college degree
5,069 (67.6)
3,385 (45.3)
College degree or more
1,439 (76.2)
1,109 (57.7)
Overall Age (years)
Sex
vaccination in a facility or ambulatory setting (mostly VA facilities or hospital outpatient departments), 70.9% of them did not have a Medicare claim, and this was fairly consistent across race/ ethnic groups.
Discussion
Marital status
Race/ethnicity
Education
Medicaid enrollment (dual eligibles) No
5,327 (70.9)
3,809 (50.4)
Yes
1,181 (62.3)
685 (36.4)
No
1,937 (66.7)
1,222 (42.8)
Yes
4,571 (70.5)
3,272 (50.1)
Metropolitan area residence
Total unweighted linked sample size, n¼9,378. Estimates are weighted to take into account the MCBS sample design. MCBS, Medicare Current Beneficiary Survey.
a
b
depending upon race/ethnicity. The next most common reported setting was within the community (32.3%), again with variation depending upon race/ethnicity. Among beneficiaries who said they received their influenza vaccination in a physician’s office, one third (33.2%) did not have a claim; for those in a community setting, one fifth (21.4%) did not have a claim. In both of these settings, non-Hispanic blacks and Hispanics were more likely to lack immunization vaccination claims. Although only 6.8% of beneficiaries reported receiving their
April 2015
387
This study compared selfreported influenza vaccination status of Medicare beneficiaries to their Medicare claims and 20.9 found that vaccination cover23.3 age levels differed in the two data sources by 420 percentage points. This result is consistent with other studies of 19.8 Medicare claims5,14,15 and med27.6 ical records.9 Using self-report 29.4 as the referent standard, Medicare claims for influenza vac28.3 cination were moderately 24.2 sensitive and highly specific compared to self-reported 22.3 information. However, the level of concordance varied by socio18.5 demographic group. In particular, the sensitivity of Medicare 20.5 claims was much lower for non-Hispanic blacks, His25.9 panics, and Asian/Pacific Islanders, as well as for low-SES 23.9 beneficiaries. Even after adjusting for sociodemographic 20.4 characteristics, beneficiaries in these groups who reported receiving the vaccine were less likely to have a Medicare claim for influenza vaccination. True vaccination status for the beneficiaries is unknown because neither the Medicare claims nor beneficiary selfreport is a clear gold standard. This study did examine whether some discordance could be attributed to beneficiaries who reported receiving an influenza vaccine, having received it outside the period of claims searched (September to December 2010). To examine this recall bias, the period of claims searched was broadened to include August 2010 to May 2011, but this did not materially affect the results. Still, the requirement for annual vaccinations poses 19.6
388
Lochner et al / Am J Prev Med 2015;48(4):384–391
Table 2. Concordance of Influenza Vaccination Between Self-Reporta and Medicare Claims: MCBS Linked Sample,b 2011, %c Overall agreement
Sensitivity
Specificity
Positive predictive value
Negative predictive value
76.3
67.5
96.3
97.6
56.6
Less than 65
71.4
50.6
96.4
94.3
61.9
65 and older
77.3
70.2
96.2
98.0
54.9
Men
73.8
63.2
96.1
97.2
55.4
Women
78.3
70.8
96.4
97.9
57.7
Married or widowed
77.2
69.9
96.5
98.1
54.8
Divorced, separated, never married
73.2
57.7
95.7
95.1
60.9
Non-Hispanic white
78.1
70.9
96.4
98.0
56.7
Non-Hispanic black
69.3
46.9
96.6
94.4
59.9
Asian/Pacific Islander
69.1
58.3
97.2
98.2
47.1
Hispanic
67.2
52.1
93.8
93.6
52.7
Other
74.3
62.3
97.7
98.1
57.0
Less than college degree
75.3
65.3
96.3
97.3
57.1
College degree or more
79.7
74.6
96.1
98.4
54.2
No
77.6
69.8
96.7
98.1
56.7
Yes
70.0
55.1
94.6
94.5
56.0
No
74.2
62.8
97.2
97.8
56.6
Yes
77.1
69.2
95.8
97.6
56.6
Overall Age
Sex
Marital status
Race/ethnicity
Education
Medicaid enrollment (dual eligibles)
Metropolitan area residence
Self-reported information is used as the “referent standard” for concordance measures. Total unweighted linked sample size, n¼9,378. c Estimates are weighted to take into account the MCBS sample design. MCBS, Medicare Current Beneficiary Survey. a
b
challenges with a beneficiary’s ability to recall exactly when the vaccination was received, including the possibility of reporting the current year’s vaccination as occurring in the prior year or vice versa. Also, limitations of the Medicare data must be considered. For example, Medicare beneficiaries who received their vaccination at a Federally Qualified Health Center (FQHC) or a Rural Health Clinic (RHC) would not have a claim in the Medicare data because influenza vaccination in these health clinics is reimbursed through the cost report. Cost reports list
aggregate totals for immunizations rather than identify individual beneficiaries. The examination of whether the presence of a claim varies by where the beneficiary reported receiving their influenza vaccination showed some interesting initial findings. About one third of beneficiaries who reported they received an influenza vaccination in a physician’s office (or medical clinic) did not have a claim, and this increased to 50% among African American beneficiaries. In addition, although only a small percentage of
www.ajpmonline.org
Lochner et al / Am J Prev Med 2015;48(4):384–391 a
Table 3. Percentage of MCBS Respondents Reporting Influenza Vaccination with a Medicare Claim for Influenza Vaccination, 2011 Mean (%)b
95% CIb
Less than 65
46.4
41.1, 51.8
65 and older
58.6
54.6, 62.6
Men
47.8
43.6, 52.1
Women
57.2
53.0, 61.4
Married or widowed
55.5
51.1, 59.8
Divorced, separated, or never married
49.6
44.7, 54.5
Non-Hispanic white
64.4
61.5, 67.1
Non-Hispanic black
43.6
36.3, 51.2
Asian/Pacific Islander
49.2
37.4, 61.1
Hispanic
46.2
40.2, 52.4
Other
58.9
51.1, 66.3
Less than college degree
48.6
44.8, 52.4
College degree or more
56.5
51.5, 61.4
53.8
48.6, 58.9
51.3
46.8, 55.8
No
48.4
43.1, 53.8
Yes
56.7
52.7, 60.5
Age (years)*
Sex
*
Marital status
Race/ethnicity*
*
Education
Medicaid enrollment (dual eligibles) No Yes Metropolitan area residence
*
a
Total unweighted linked sample with self-report of influenza vaccination, n¼6,508. b Predicted marginal means estimated from a logistic regression model. Estimates and standard errors take into account the MCBS sample design. n po0.01. MCBS, Medicare Current Beneficiary Survey.
beneficiaries reported receiving vaccination from a VA facility or other freestanding ambulatory care setting, 70% of beneficiaries did not have a Medicare claim. Such discrepancies may indicate over-reporting in the survey, or they may suggest free influenza vaccine events at VA facilities or other settings. Studies that compare selfreport, Medicare claims, and medical records or chart evaluation could provide greater validity for both selfreport and Medicare claims. April 2015
389
Finally, although this study was not able to determine the exact causes of discordance, the findings point to several areas that warrant further investigation to better understand the potential misclassification of vaccination status. There is the possibility that beneficiaries are paying out of pocket for vaccination in community settings, although it is a covered benefit with no copay. The recently released (July 2014) 2012 MCBS Access to Care File has added a question about out of pocket payment for influenza vaccines (www.cms.gov/ResearchStatistics-Data-and-Systems/Research/MCBS/index.html? redirect=/mcbs). Fewer claims among African American, Hispanic, and low-SES beneficiaries who reported receiving the vaccination may indicate issues with provider billing, or it may indicate over-reporting in the survey. Although there have been several validation studies of self-reported influenza vaccination,8–12 it also has been estimated that self-reported influenza vaccination may overstate coverage by about 10%11; few have examined how validity may differ by sociodemographic characteristics. Studies that addressed this issue have found that African Americans and Hispanics, as well as people with lower SES, were more likely to over-report receiving vaccinations.16,17 These studies, however, were not focused on Medicare populations, and only one included influenza vaccination. As studies have found more negative attitudes and concern about influenza vaccination among African Americans and Hispanics,14,18–20 additional research on response bias is needed that examines negative attitudes and beliefs, self-reported vaccination status, and medical records or claims. This issue could not be addressed in this study because the MCBS only asked about attitudes and beliefs toward vaccination for beneficiaries not reporting an influenza vaccination. Although influenza vaccination coverage is routinely based upon self-reported influenza vaccination status, Medicare claims are increasingly being used for such surveillance. These findings, among a sample of Medicare beneficiaries with linked information on self-report and Medicare claims, demonstrate the magnitude of the discrepancy between claims-based vaccination status and self-report. Importantly, the findings identify groups of Medicare beneficiaries for whom the discrepancy is greatest, which can inform influenza vaccination surveillance studies, particularly those focused on disparities in coverage by race/ethnicity or geography. Finally, these findings point to several areas where additional research is needed to understand the strengths and limitations of each data source so that factors associated with discordance can be identified, specifically for sensitivity and NPV, leading to improved vaccination assessment data used for surveillance.
75.4 73.0
—e 55.8 55.5 2.5d 4.7d
3.7
3.4 Otherf
3.6 3.7 Clinics/centers/health department
3.3
64.3 22.0 27.9 —
69.6 70.9 6.8 Facility/ambulatory treatment center
6.1
10.9
d
e
9.6
19.6 21.4 26.1 15.2 32.3 Community setting
34.4
65.4 52.6 53.9 Physician office/practice
Estimates are weighted to reflect the MCBS sample design. Total unweighted linked sample with self-report of influenza vaccination, n¼6,508. Columns may not total 100% because of rounding. Values reflect percentages within the service setting and do not total 100%. For the final four columns, the denominators are the total number of beneficiaries who reported receiving a vaccination in the particular setting. d Relative standard error (RSE) of the estimate is 420%. e Suppressed because relative standard error (RSE) of the estimate is 430%. f Other Place of Service includes responses of work/employment, managed care insurer, other, and unknown. MCBS, Medicare Current Beneficiary Survey. c
b
a
29.8 48.6
—e
50.0 49.6 57.4
33.2
29.5
d
Hispanics Non-Hispanic blacks Non-Hispanic whites All beneficiaries Hispanics Non-Hispanic blacks Non-Hispanic whites All beneficiaries
No Medicare claim among beneficiaries reporting vaccination in the settingc Beneficiaries reporting influenza vaccination in the settingb
Table 4. MCBS Respondents’ Reported Place of Service for Receiving Influenza Vaccination, 2011, %a
—e
Lochner et al / Am J Prev Med 2015;48(4):384–391
390
The authors are with the Centers for Medicare & Medicaid Services (CMS). The views and opinions expressed in this article are those of the authors and do not necessarily reflect the views of CMS. No authors have any conflicts of interest with the study, as all authors are employees of CMS, used only CMS data, and required no external funding for the research. No financial disclosures were reported by the authors of this paper.
References 1. CDC. Prevention and control of seasonal influenza with vaccines. Recommendations of the Advisory Committee on Immunization Practices—United States, 2013-2014. MMWR Recomm Rep 2013;62 (RR-07):1–43. 2. CDC. Surveillance of influenza vaccination coverage—United States, 2007-08 through 2011-12 influenza seasons. MMWR Surveill Summ 2013;62(4):1–28. 3. Pavlin JA, Mostashari F, Kortepeter MG, et al. Innovative surveillance methods for rapid detection of disease outbreaks and bioterrorism: results of an interagency workshop on health indicator surveillance. Am J Public Health. 2003;93(8):1230–5. http://dx.doi.org/10.2105/ AJPH.93.8.1230. 4. Burwen DR, Sandhu SK, MaCurdy TE, et al. Surveillance for GuillainBarré syndrome after influenza vaccination among the Medicare population, 2009-2010. Am J Public Health. 2012;102(10):1921–7. http://dx.doi.org/10.2105/AJPH.2011.300510. 5. Burwen DR, La Voie L, Braun MM, Houck P, Ball R. Evaluating adverse events after vaccination in the Medicare population. Pharmacoepidemiol Drug Saf. 2007;16(7):753–61. http://dx.doi.org/10. 1002/pds.1390. 6. Burwen DR, Ball R, Bryan WW, et al. Evaluation of Guillain-Barré syndrome among recipients of influenza vaccine in 2000 and 2001. Am J Prev Med 2010;39(4):296–304. http://dx.doi.org/10.1016/j.amepre. 2010.05.022. 7. CDC. Influenza vaccination status of persons aged 65-79 years— Allegheny County, Pennsylvania, February-March 1997. MMWR 1998;47(50):1094–7. 8. Mac Donald R, Baken L, Nelson A, Nichol KL. Validation of self-report of influenza and pneumococcal vaccination status in elderly outpatients. Am J Prev Med. 1999;16(3):173–7. http://dx.doi.org/10.1016/ S0749-3797(98)00159-7. 9. Zimmerman RK, Raymund M, Janosky JE, et al. Sensitivity and specificity of patient self-report of influenza and pneumococcal polysaccharide vaccinations among elderly outpatients in diverse patient care strata. Vaccine 2003;21(13/14):1486–91. http://dx.doi.org/ 10.1016/S0264-410X(02)00700-4. 10. Mangtani P, Shah A, Roberts JA. Validation of influenza and pneumococcal vaccine status in adults based on self-report. Epidemiol Infect. 2007;135(1):139–43. http://dx.doi.org/10.1017/S0950268806006479. 11. Skull SA, Andrews RM, Byrnes GB, et al. Validity of self-reported influenza and pneumococcal vaccination status among a cohort of hospitalized elderly inpatients. Vaccine 2007;25(25):4775–83. http:// dx.doi.org/10.1016/j.vaccine.2007.04.015. 12. Irving SA, Donahue JG, Shay DK, Ellis-Coyle TL, Belongia EA. Evaluation of self-reported and registry-based influenza vaccination status in a Wisconsin cohort. Vaccine 2009;27(47):6546–9. http:// dx.doi.org/10.1016/j.vaccine.2009.08.050. 13. Heeringa SG, West BT, Berglund PA. Applied Survey Data Analysis. Boca Raton, FL: Chapman & Hall/CRC Statistics in the Social and Behavioral Sciences, 2010. http://dx.doi.org/10.1201/9781420080674.
www.ajpmonline.org
Lochner et al / Am J Prev Med 2015;48(4):384–391 14. Hebert PL, Frick KD, Kane RL, McBean M. The causes of racial and ethnic differences in influenza vaccination rates among elderly Medicare beneficiaries. Health Serv Res. 2005;40(2):517–37. http:// dx.doi.org/10.1111/j.1475-6773.2005.0e371.x. 15. Fiscella K, Holt K, Meldrum S, Franks P. Disparities in preventive procedures: comparisons of self-report and Medicare claims data. Health Serv Res. 2006;6:122. http://dx.doi.org/10.1186/1472-6963-6-122. 16. Gordon NP, Wortley PM, Singleton JA, Lin TY, Bardenheier BH. Race/ ethnicity and validity of self-reported pneumococcal vaccination. BMC Public Health. 2008;8:227. http://dx.doi.org/10.1186/1471-2458-8-227. 17. Rolnick SJ, Parker ED, Nordin JD, et al. Self-report compared to electronic medical record across eight adult vaccines: do results vary by
April 2015
391
demographic factors? Vaccine 2013;31(37):3928–35. http://dx.doi.org/ 10.1016/j.vaccine.2013.06.041. 18. Lindley MC, Wortley PM, Winston CA, Bardenheier BH. The role of attitudes in understanding disparities in adult influenza vaccination. Am J Prev Med. 2006;31(4):281–5. http://dx.doi.org/10.1016/j.amepre.2006.06.025. 19. Lochner KA, Wynne M. Flu shots and the characteristics of unvaccinated elderly Medicare beneficiaries. Medicare Medicaid Res Rev 2011;1(4):E1–E11. http://dx.doi.org/10.5600/mmrr.001.04.b01. 20. Wooten KG, Wortley PM, Singleton JA, Euler GL. Perceptions matter: beliefs about influenza vaccine and vaccination behavior among elderly white, black and Hispanic Americans. Vaccine 2012;30(48):6927–34. http://dx.doi.org/10.1016/j.vaccine.2012.08.036.
Purpose: To evaluate the concordance between self-reported influenza vaccination and influenza vaccination claims among Medicare beneficiaries.
Methods: This study compared influenza vaccination based upon Medicare claims and self-report among a sample of Medicare beneficiaries (N¼9,378) from the 2011 Medicare Current Beneficiary Survey, which was the most recent year of data at the time of analysis (summer 2013). Sensitivity, specificity, positive predictive value, and negative predictive value were calculated using selfreported data as the referent standard. Logistic regression was used to compute the marginal mean proportions for whether a Medicare influenza vaccination claim was present among beneficiaries who reported receiving the vaccination.
Results: Influenza vaccination was higher for self-report (69.4%) than Medicare claims (48.3%). For Medicare claims, sensitivity¼67.5%, specificity¼96.3%, positive predictive value¼97.6%, and negative predictive value¼56.7%. Among beneficiaries reporting receiving an influenza vaccination, the percentage of beneficiaries with a vaccination claim was lower for beneficiaries who were aged o65 years, male, non-Hispanic black or Hispanic, and had less than a college education. Conclusions: The classification of influenza vaccination status for Medicare beneficiaries can differ based upon survey and claims. To improve Medicare claims–based surveillance studies, further research is needed to determine the sources of discordance in self-reported and Medicare claims data, specifically for sensitivity and negative predictive value. (Am J Prev Med 2015;48(4):384–391) Published by Elsevier Inc. on behalf of American Journal of Preventive Medicine
Introduction
A
n annual influenza vaccination is recommended for all people aged Z6 months in the U.S. who do not have contraindications to the vaccine.1 Surveillance of influenza vaccination coverage is essential in determining season-specific estimates and trends as well as From the Centers for Medicare & Medicaid Services, Sam Nunn Atlanta Federal Center (Lochner), Atlanta, Georgia; Office of Information Products and Data Analytics (Lochner, Wheatcroft); and Center for Medicare (Wynne, Worrall, Kelman), Baltimore, Maryland Address correspondence to: Kimberly A. Lochner, ScD, CMS, Sam Nunn Atlanta Federal Center, 61 Forsyth Street, S.W., Suite 4T20, Atlanta GA 30303. E-mail: [email protected]. 0749-3797/$36.00 http://dx.doi.org/10.1016/j.amepre.2014.10.016
384
Am J Prev Med 2015;48(4):384–391
identifying groups with lower coverage by geographic area, age, race/ethnicity, and SES.2 Although self-reported influenza vaccination status, collected in national surveys, is routinely used in surveillance to estimate influenza vaccine coverage across the U.S. (www.cdc.gov/flu/fluvaxview/ index.htmh), Medicare data are becoming a promising resource for influenza surveillance to inform vaccination program management and planning. Medicare claims data cover a large proportion of the population aged Z65 years, a population at high risk for complications due to influenza. Medicare claims data have other advantages, such as the ability to generate small-area estimates, as well as near “realtime” seasonal information.3,4 Medicare claims have been used to monitor associations between influenza vaccinations and rare adverse
Published by Elsevier Inc. on behalf of American Journal of Preventive Medicine
Lochner et al / Am J Prev Med 2015;48(4):384–391 4–6
events and recently to track influenza vaccination coverage and disparities in coverage for the Medicare population (www.hhs.gov/nvpo/flu-vaccination-map/ index.html). Yet, compared with other data sources that assess influenza vaccination coverage, those based upon Medicare claims submitted for reimbursement for influenza vaccine are substantially lower. A study of people aged 65–79 years in Allegheny County PA found that influenza vaccination was estimated to be 75% based upon self-report and only 36% based upon Medicare claims.7 Influenza vaccination coverage based upon several surveys was 65%–70%, whereas coverage based upon Medicare claims was just more than 40%.5 Because of the substantial discrepancy between Medicare claims and survey self-report, studies of influenza surveillance may be hindered by the potential misclassification of a beneficiary’s influenza vaccination status depending upon the data source. Direct comparisons between claims and self-report among the same Medicare beneficiaries are needed to better understand the strengths and limitations of specific data sources so that the data used to assess vaccination status can be improved. This study directly compared Medicare beneficiaries’ claims and their self-reported information using a sample of respondents to the Medicare Current Beneficiary Survey (MCBS) who were linked to their Medicare enrollment and claims information. The goal of this study was to assess the concordance of Medicare claims for influenza vaccination and self-reported vaccination status, quantify the sensitivity of ascertainment of vaccination in the claims, and determine the specific populations for whom the discordance is greatest. This study also explores where beneficiaries reported receiving the influenza vaccination as a means to help identify possible reasons for the discordance.
Methods Data Source and Study Population Medicare is the U.S. federal health insurance program for individuals aged Z65 years, those aged o65 years with certain disabilities, and those of any age with end-stage renal disease (ESRD). To examine the concordance between Medicare beneficiaries’ self-report of influenza vaccination and their Medicare claims, information from two data sources available from the Centers for Medicare & Medicaid Services (CMS) were linked. The first source was the MCBS, a continuous, multipurpose survey of a nationally representative sample of elderly, disabled, and institutionalized Medicare beneficiaries. The MCBS collects information on sociodemographic characteristics and health status of Medicare beneficiaries, as well as their use of and satisfaction with their health care (www.resdac.org/cms-data/files/mcbs-access-care). Information on influenza vaccination came from the 2011 MCBS April 2015
385
Access to Care file fall round interviews, which was the most recent year of data at the time of analysis (summer 2013). The second source was Medicare claims data available from the Chronic Conditions Warehouse (CCW), a research database with 100% of Medicare enrollment and fee-for-service institutional and non-institutional claims data (www.ccwdata.org). The use of these secondary, existing data sources was exempt from IRB approval as the participants in the MCBS provided informed consent; the data were de-identified and pose no risk to loss of confidentiality. The study population included respondents who were community-dwelling, survived the entire year (2011), responded yes or no to the question asking whether they had received an influenza vaccine, and had non-missing values for the sociodemographic variables of interest. MCBS respondents meeting these criteria were linked to the Medicare administrative data so that their claims could be searched for influenza vaccination codes for the period from September to December 2010, as this is the reference period for the influenza question on the MCBS. Because claims data are not available for beneficiaries enrolled in Medicare Advantage, the linked MCBS respondents had to be continuously enrolled in fee-for-service Medicare parts A and B for the period September to December 2010, which was determined from the Medicare enrollment data. This resulted in a study population of 9,378, representing approximately 27.8 million fee-for-service Medicare beneficiaries. This linkage at the beneficiary level allowed for direct comparisons between beneficiaries’ self-reported influenza vaccination and vaccination documented by the presence of Medicare claims.
Influenza Measures The 2011 MCBS Access to Care file, which was fielded during fall 2011, asked beneficiaries if they had received a seasonal influenza vaccine for the prior year, during the period September to December 2010. Medicare beneficiaries’ self-reported (yes or no) influenza vaccination status was based upon this response. For these same beneficiaries, their institutional and non-institutional claims for September to December 2010 were examined. An influenza vaccination was identified in the Medicare claims by the following Healthcare Common Procedure Coding System (HCPCS) codes: 90655–90658, 90660, 90662, Q2035–Q2039, and G0008. In addition, to address the availability of influenza vaccinations before September and after December, and given that beneficiaries may not accurately recall when they received the vaccination, a secondary analysis broadened the period of claims searched to include August 2010 to May 2011. Only 3.5% (n¼165) of beneficiaries in the linked study sample with an influenza vaccination claim had a claim outside September to December 2010, which did not affect the results.
Medicare Beneficiary Characteristics and SelfReported Place of Service for Influenza Vaccination The study examined standard sociodemographic characteristics of the study population available from the MCBS: age (categorized as o65 years and Z65 years); sex; marital status (married/widowed or divorced/separated/never married); education (less than a college degree and college degree or more); metropolitan residence (versus not); dual eligibility for Medicaid benefits (versus not); and race/ ethnicity (non-Hispanic white, non-Hispanic black or African American, Hispanic, Asian or Pacific Islander, and all other races).
386
Lochner et al / Am J Prev Med 2015;48(4):384–391
For those MCBS respondents who reported that they received an influenza vaccination, a follow-up question asked where they went for their vaccine. These responses were coded into 21 categories, which were then collapsed into five categories: (1) physician/doctor office; (2) community setting (e.g., health fair, shopping mall/store/pharmacy, senior center, church, school); (3) facility/ambulatory treatment center (e.g., freestanding surgical centers, urgent care centers, emergency department, Veterans Affairs [VA] facility); (4) clinics (e.g., rural clinics, neighborhood health centers) or health department; and (5) all others, which, because of small sample sizes, includes place of work or employment, managed care plan center or HMO, or unknown. Also owing to small sample sizes, this information is presented only for non-Hispanic whites, non-Hispanic blacks, and Hispanics.
Statistical Analysis First, the study examined the concordance between Medicare claims with influenza vaccination HCPCS or procedure codes and selfreported influenza vaccination in the sample of MCBS respondents linked to their claims data. As several studies have demonstrated selfreported influenza vaccination to be a highly sensitive (Z95%) and variably specific (38%–90%) indicator of actual vaccine status compared to medical records,8–12 the MCBS self-reported information was used as the referent standard. Overall agreement between the two data sources as well as the sensitivity; specificity; positive predictive value (PPV); and negative predictive value (NPV) of the Medicare claims data was examined. Overall agreement was calculated as the percentage of true positives and true negatives, meaning agreement from both data sources, out of the entire study population. Sensitivity refers to the probability of the presence of an influenza vaccination claim if there is self-reported influenza vaccination; specificity refers to the probability of the absence of an influenza vaccination claim if there is no self-reported influenza vaccination. PPV refers to the probability that there is self-reported influenza vaccination given that an influenza vaccination claim is present, and NPV refers to the probability that there is no self-reported influenza vaccination given that there is no influenza vaccination claim present. The study also examined factors associated with a beneficiary having a Medicare claim for influenza vaccination among beneficiaries who reported receiving the vaccination. Using a logistic regression model that included the beneficiary characteristics described above as the predictors, the marginal mean proportions were computed for whether a Medicare influenza vaccination claim was present among beneficiaries who reported receiving the vaccination. Finally, the study described where beneficiaries reported they received the influenza vaccination, using the place of service categories described above, and whether a Medicare claim was present. To take into account the complex survey design of the MCBS, all analyses were conducted using SAS, version 9.3 SURVEY procedures. Sample weights, which account for the differential probabilities of selection, non-response, and non-coverage, were incorporated into the estimation process. The SEs of the percentages were estimated using balanced repeated replicates (BRR), a method that incorporates the sample weights and sample design.13
Results The study population was composed mainly of Medicare beneficiaries who were aged Z65 years (82.3%); women
(55.2%); non-Hispanic whites (79.2%); and those living in a metropolitan area (71.8%). Approximately one fifth had at least a college degree (21.7%), and 17.2% were eligible for Medicaid benefits (“dual eligibles”; data not shown). Table 1 shows the percentage of Medicare beneficiaries receiving an influenza vaccination based upon source (self-report versus Medicare claims). Influenza vaccination coverage was 69.4% based upon selfreport and 48.3% based upon Medicare claims. Coverage was lower based upon claims data compared to survey data for all groups, with the largest differences (425 percentage points) observed for beneficiaries aged o65 years of age; non-Hispanic blacks, Hispanics, or Asian/ Pacific Islanders; and those also enrolled in Medicaid (dual-eligibles). Table 2 presents the results of the concordance between the two data sources overall and by sociodemographic characteristics. Overall, the agreement between the two data sources was 76.3% and ranged from 79.7% (for beneficiaries with at least a college degree) to 67.2% (for Hispanic beneficiaries). The sensitivity of the Medicare claims was 67.5% and the specificity was 96.3%; PPV was 97.6% and NPV was 56.6%. Sensitivity was 50.6% for beneficiaries aged o65 years and 70.2% for those aged Z65 years; it was higher for women (70.8%) than men (63.2%), highest for non-Hispanic whites (70.9%), and lowest for non-Hispanic blacks (46.9%). Sensitivity also was lower for those with less than a college degree (65.3%) and dual-eligible beneficiaries (55.1%). Specificity and PPV were high for all groups (at least 94%), whereas NPV was lower than 60% for most groups and lowest for Asian/Pacific Islanders (47.1%). Table 3 presents the predicted marginal mean proportions of having a Medicare influenza vaccination claim, controlling for all variables examined in the logistic model, among Medicare beneficiaries reporting they received the influenza vaccination. Age, sex, marital status, race/ethnicity, education level, and urban/rural residence were statistically significant (po0.01) predictors of whether a Medicare influenza vaccination claim was present. The percentage of beneficiaries with a claim was lower for beneficiaries who were aged o65 years, male, and non-Hispanic black or Hispanic and had less than a college education. The lowest percentage was among non-Hispanic blacks, where 43.6% who reported receiving the vaccination had a claim. Table 4 presents descriptive findings examining where beneficiaries reported they received the influenza vaccination and whether a Medicare claim was present. Among beneficiaries who reported receiving an influenza vaccination, more than half (53.9%) reported receiving it in a physician’s office, varying from 53% to 65% www.ajpmonline.org
Lochner et al / Am J Prev Med 2015;48(4):384–391 a
Table 1. Influenza Vaccination Coverage: MCBS-Linked Sample, 2011 Self-report (n [%]b)
Medicare claims (n [%]b)
Difference: selfreport – claims
6,508 (69.4)
4,494 (48.3)
21.1
Less than 65
955 (54.5)
497 (29.2)
25.3
65 and older
5,553 (72.6)
3,997 (52.0)
20.6
Men
2,868 (67.8)
1,874 (44.1)
23.7
Women
3,640 (70.8)
2,620 (51.2)
Married or widowed
5,163 (72.6)
3,693 (51.7)
Divorced, separated, never married
1,345 (59.2)
801 (35.9)
Non-Hispanic white
5,304 (71.6)
3,846 (51.8)
Non-Hispanic black
514 (54.9)
255 (27.3)
Asian/Pacific Islander
106 (72.4)
68 (43.0)
Hispanic
384 (63.7)
205 (35.4)
Other
200 (66.2)
120 (42.0)
Less than college degree
5,069 (67.6)
3,385 (45.3)
College degree or more
1,439 (76.2)
1,109 (57.7)
Overall Age (years)
Sex
vaccination in a facility or ambulatory setting (mostly VA facilities or hospital outpatient departments), 70.9% of them did not have a Medicare claim, and this was fairly consistent across race/ ethnic groups.
Discussion
Marital status
Race/ethnicity
Education
Medicaid enrollment (dual eligibles) No
5,327 (70.9)
3,809 (50.4)
Yes
1,181 (62.3)
685 (36.4)
No
1,937 (66.7)
1,222 (42.8)
Yes
4,571 (70.5)
3,272 (50.1)
Metropolitan area residence
Total unweighted linked sample size, n¼9,378. Estimates are weighted to take into account the MCBS sample design. MCBS, Medicare Current Beneficiary Survey.
a
b
depending upon race/ethnicity. The next most common reported setting was within the community (32.3%), again with variation depending upon race/ethnicity. Among beneficiaries who said they received their influenza vaccination in a physician’s office, one third (33.2%) did not have a claim; for those in a community setting, one fifth (21.4%) did not have a claim. In both of these settings, non-Hispanic blacks and Hispanics were more likely to lack immunization vaccination claims. Although only 6.8% of beneficiaries reported receiving their
April 2015
387
This study compared selfreported influenza vaccination status of Medicare beneficiaries to their Medicare claims and 20.9 found that vaccination cover23.3 age levels differed in the two data sources by 420 percentage points. This result is consistent with other studies of 19.8 Medicare claims5,14,15 and med27.6 ical records.9 Using self-report 29.4 as the referent standard, Medicare claims for influenza vac28.3 cination were moderately 24.2 sensitive and highly specific compared to self-reported 22.3 information. However, the level of concordance varied by socio18.5 demographic group. In particular, the sensitivity of Medicare 20.5 claims was much lower for non-Hispanic blacks, His25.9 panics, and Asian/Pacific Islanders, as well as for low-SES 23.9 beneficiaries. Even after adjusting for sociodemographic 20.4 characteristics, beneficiaries in these groups who reported receiving the vaccine were less likely to have a Medicare claim for influenza vaccination. True vaccination status for the beneficiaries is unknown because neither the Medicare claims nor beneficiary selfreport is a clear gold standard. This study did examine whether some discordance could be attributed to beneficiaries who reported receiving an influenza vaccine, having received it outside the period of claims searched (September to December 2010). To examine this recall bias, the period of claims searched was broadened to include August 2010 to May 2011, but this did not materially affect the results. Still, the requirement for annual vaccinations poses 19.6
388
Lochner et al / Am J Prev Med 2015;48(4):384–391
Table 2. Concordance of Influenza Vaccination Between Self-Reporta and Medicare Claims: MCBS Linked Sample,b 2011, %c Overall agreement
Sensitivity
Specificity
Positive predictive value
Negative predictive value
76.3
67.5
96.3
97.6
56.6
Less than 65
71.4
50.6
96.4
94.3
61.9
65 and older
77.3
70.2
96.2
98.0
54.9
Men
73.8
63.2
96.1
97.2
55.4
Women
78.3
70.8
96.4
97.9
57.7
Married or widowed
77.2
69.9
96.5
98.1
54.8
Divorced, separated, never married
73.2
57.7
95.7
95.1
60.9
Non-Hispanic white
78.1
70.9
96.4
98.0
56.7
Non-Hispanic black
69.3
46.9
96.6
94.4
59.9
Asian/Pacific Islander
69.1
58.3
97.2
98.2
47.1
Hispanic
67.2
52.1
93.8
93.6
52.7
Other
74.3
62.3
97.7
98.1
57.0
Less than college degree
75.3
65.3
96.3
97.3
57.1
College degree or more
79.7
74.6
96.1
98.4
54.2
No
77.6
69.8
96.7
98.1
56.7
Yes
70.0
55.1
94.6
94.5
56.0
No
74.2
62.8
97.2
97.8
56.6
Yes
77.1
69.2
95.8
97.6
56.6
Overall Age
Sex
Marital status
Race/ethnicity
Education
Medicaid enrollment (dual eligibles)
Metropolitan area residence
Self-reported information is used as the “referent standard” for concordance measures. Total unweighted linked sample size, n¼9,378. c Estimates are weighted to take into account the MCBS sample design. MCBS, Medicare Current Beneficiary Survey. a
b
challenges with a beneficiary’s ability to recall exactly when the vaccination was received, including the possibility of reporting the current year’s vaccination as occurring in the prior year or vice versa. Also, limitations of the Medicare data must be considered. For example, Medicare beneficiaries who received their vaccination at a Federally Qualified Health Center (FQHC) or a Rural Health Clinic (RHC) would not have a claim in the Medicare data because influenza vaccination in these health clinics is reimbursed through the cost report. Cost reports list
aggregate totals for immunizations rather than identify individual beneficiaries. The examination of whether the presence of a claim varies by where the beneficiary reported receiving their influenza vaccination showed some interesting initial findings. About one third of beneficiaries who reported they received an influenza vaccination in a physician’s office (or medical clinic) did not have a claim, and this increased to 50% among African American beneficiaries. In addition, although only a small percentage of
www.ajpmonline.org
Lochner et al / Am J Prev Med 2015;48(4):384–391 a
Table 3. Percentage of MCBS Respondents Reporting Influenza Vaccination with a Medicare Claim for Influenza Vaccination, 2011 Mean (%)b
95% CIb
Less than 65
46.4
41.1, 51.8
65 and older
58.6
54.6, 62.6
Men
47.8
43.6, 52.1
Women
57.2
53.0, 61.4
Married or widowed
55.5
51.1, 59.8
Divorced, separated, or never married
49.6
44.7, 54.5
Non-Hispanic white
64.4
61.5, 67.1
Non-Hispanic black
43.6
36.3, 51.2
Asian/Pacific Islander
49.2
37.4, 61.1
Hispanic
46.2
40.2, 52.4
Other
58.9
51.1, 66.3
Less than college degree
48.6
44.8, 52.4
College degree or more
56.5
51.5, 61.4
53.8
48.6, 58.9
51.3
46.8, 55.8
No
48.4
43.1, 53.8
Yes
56.7
52.7, 60.5
Age (years)*
Sex
*
Marital status
Race/ethnicity*
*
Education
Medicaid enrollment (dual eligibles) No Yes Metropolitan area residence
*
a
Total unweighted linked sample with self-report of influenza vaccination, n¼6,508. b Predicted marginal means estimated from a logistic regression model. Estimates and standard errors take into account the MCBS sample design. n po0.01. MCBS, Medicare Current Beneficiary Survey.
beneficiaries reported receiving vaccination from a VA facility or other freestanding ambulatory care setting, 70% of beneficiaries did not have a Medicare claim. Such discrepancies may indicate over-reporting in the survey, or they may suggest free influenza vaccine events at VA facilities or other settings. Studies that compare selfreport, Medicare claims, and medical records or chart evaluation could provide greater validity for both selfreport and Medicare claims. April 2015
389
Finally, although this study was not able to determine the exact causes of discordance, the findings point to several areas that warrant further investigation to better understand the potential misclassification of vaccination status. There is the possibility that beneficiaries are paying out of pocket for vaccination in community settings, although it is a covered benefit with no copay. The recently released (July 2014) 2012 MCBS Access to Care File has added a question about out of pocket payment for influenza vaccines (www.cms.gov/ResearchStatistics-Data-and-Systems/Research/MCBS/index.html? redirect=/mcbs). Fewer claims among African American, Hispanic, and low-SES beneficiaries who reported receiving the vaccination may indicate issues with provider billing, or it may indicate over-reporting in the survey. Although there have been several validation studies of self-reported influenza vaccination,8–12 it also has been estimated that self-reported influenza vaccination may overstate coverage by about 10%11; few have examined how validity may differ by sociodemographic characteristics. Studies that addressed this issue have found that African Americans and Hispanics, as well as people with lower SES, were more likely to over-report receiving vaccinations.16,17 These studies, however, were not focused on Medicare populations, and only one included influenza vaccination. As studies have found more negative attitudes and concern about influenza vaccination among African Americans and Hispanics,14,18–20 additional research on response bias is needed that examines negative attitudes and beliefs, self-reported vaccination status, and medical records or claims. This issue could not be addressed in this study because the MCBS only asked about attitudes and beliefs toward vaccination for beneficiaries not reporting an influenza vaccination. Although influenza vaccination coverage is routinely based upon self-reported influenza vaccination status, Medicare claims are increasingly being used for such surveillance. These findings, among a sample of Medicare beneficiaries with linked information on self-report and Medicare claims, demonstrate the magnitude of the discrepancy between claims-based vaccination status and self-report. Importantly, the findings identify groups of Medicare beneficiaries for whom the discrepancy is greatest, which can inform influenza vaccination surveillance studies, particularly those focused on disparities in coverage by race/ethnicity or geography. Finally, these findings point to several areas where additional research is needed to understand the strengths and limitations of each data source so that factors associated with discordance can be identified, specifically for sensitivity and NPV, leading to improved vaccination assessment data used for surveillance.
75.4 73.0
—e 55.8 55.5 2.5d 4.7d
3.7
3.4 Otherf
3.6 3.7 Clinics/centers/health department
3.3
64.3 22.0 27.9 —
69.6 70.9 6.8 Facility/ambulatory treatment center
6.1
10.9
d
e
9.6
19.6 21.4 26.1 15.2 32.3 Community setting
34.4
65.4 52.6 53.9 Physician office/practice
Estimates are weighted to reflect the MCBS sample design. Total unweighted linked sample with self-report of influenza vaccination, n¼6,508. Columns may not total 100% because of rounding. Values reflect percentages within the service setting and do not total 100%. For the final four columns, the denominators are the total number of beneficiaries who reported receiving a vaccination in the particular setting. d Relative standard error (RSE) of the estimate is 420%. e Suppressed because relative standard error (RSE) of the estimate is 430%. f Other Place of Service includes responses of work/employment, managed care insurer, other, and unknown. MCBS, Medicare Current Beneficiary Survey. c
b
a
29.8 48.6
—e
50.0 49.6 57.4
33.2
29.5
d
Hispanics Non-Hispanic blacks Non-Hispanic whites All beneficiaries Hispanics Non-Hispanic blacks Non-Hispanic whites All beneficiaries
No Medicare claim among beneficiaries reporting vaccination in the settingc Beneficiaries reporting influenza vaccination in the settingb
Table 4. MCBS Respondents’ Reported Place of Service for Receiving Influenza Vaccination, 2011, %a
—e
Lochner et al / Am J Prev Med 2015;48(4):384–391
390
The authors are with the Centers for Medicare & Medicaid Services (CMS). The views and opinions expressed in this article are those of the authors and do not necessarily reflect the views of CMS. No authors have any conflicts of interest with the study, as all authors are employees of CMS, used only CMS data, and required no external funding for the research. No financial disclosures were reported by the authors of this paper.
References 1. CDC. Prevention and control of seasonal influenza with vaccines. Recommendations of the Advisory Committee on Immunization Practices—United States, 2013-2014. MMWR Recomm Rep 2013;62 (RR-07):1–43. 2. CDC. Surveillance of influenza vaccination coverage—United States, 2007-08 through 2011-12 influenza seasons. MMWR Surveill Summ 2013;62(4):1–28. 3. Pavlin JA, Mostashari F, Kortepeter MG, et al. Innovative surveillance methods for rapid detection of disease outbreaks and bioterrorism: results of an interagency workshop on health indicator surveillance. Am J Public Health. 2003;93(8):1230–5. http://dx.doi.org/10.2105/ AJPH.93.8.1230. 4. Burwen DR, Sandhu SK, MaCurdy TE, et al. Surveillance for GuillainBarré syndrome after influenza vaccination among the Medicare population, 2009-2010. Am J Public Health. 2012;102(10):1921–7. http://dx.doi.org/10.2105/AJPH.2011.300510. 5. Burwen DR, La Voie L, Braun MM, Houck P, Ball R. Evaluating adverse events after vaccination in the Medicare population. Pharmacoepidemiol Drug Saf. 2007;16(7):753–61. http://dx.doi.org/10. 1002/pds.1390. 6. Burwen DR, Ball R, Bryan WW, et al. Evaluation of Guillain-Barré syndrome among recipients of influenza vaccine in 2000 and 2001. Am J Prev Med 2010;39(4):296–304. http://dx.doi.org/10.1016/j.amepre. 2010.05.022. 7. CDC. Influenza vaccination status of persons aged 65-79 years— Allegheny County, Pennsylvania, February-March 1997. MMWR 1998;47(50):1094–7. 8. Mac Donald R, Baken L, Nelson A, Nichol KL. Validation of self-report of influenza and pneumococcal vaccination status in elderly outpatients. Am J Prev Med. 1999;16(3):173–7. http://dx.doi.org/10.1016/ S0749-3797(98)00159-7. 9. Zimmerman RK, Raymund M, Janosky JE, et al. Sensitivity and specificity of patient self-report of influenza and pneumococcal polysaccharide vaccinations among elderly outpatients in diverse patient care strata. Vaccine 2003;21(13/14):1486–91. http://dx.doi.org/ 10.1016/S0264-410X(02)00700-4. 10. Mangtani P, Shah A, Roberts JA. Validation of influenza and pneumococcal vaccine status in adults based on self-report. Epidemiol Infect. 2007;135(1):139–43. http://dx.doi.org/10.1017/S0950268806006479. 11. Skull SA, Andrews RM, Byrnes GB, et al. Validity of self-reported influenza and pneumococcal vaccination status among a cohort of hospitalized elderly inpatients. Vaccine 2007;25(25):4775–83. http:// dx.doi.org/10.1016/j.vaccine.2007.04.015. 12. Irving SA, Donahue JG, Shay DK, Ellis-Coyle TL, Belongia EA. Evaluation of self-reported and registry-based influenza vaccination status in a Wisconsin cohort. Vaccine 2009;27(47):6546–9. http:// dx.doi.org/10.1016/j.vaccine.2009.08.050. 13. Heeringa SG, West BT, Berglund PA. Applied Survey Data Analysis. Boca Raton, FL: Chapman & Hall/CRC Statistics in the Social and Behavioral Sciences, 2010. http://dx.doi.org/10.1201/9781420080674.
www.ajpmonline.org
Lochner et al / Am J Prev Med 2015;48(4):384–391 14. Hebert PL, Frick KD, Kane RL, McBean M. The causes of racial and ethnic differences in influenza vaccination rates among elderly Medicare beneficiaries. Health Serv Res. 2005;40(2):517–37. http:// dx.doi.org/10.1111/j.1475-6773.2005.0e371.x. 15. Fiscella K, Holt K, Meldrum S, Franks P. Disparities in preventive procedures: comparisons of self-report and Medicare claims data. Health Serv Res. 2006;6:122. http://dx.doi.org/10.1186/1472-6963-6-122. 16. Gordon NP, Wortley PM, Singleton JA, Lin TY, Bardenheier BH. Race/ ethnicity and validity of self-reported pneumococcal vaccination. BMC Public Health. 2008;8:227. http://dx.doi.org/10.1186/1471-2458-8-227. 17. Rolnick SJ, Parker ED, Nordin JD, et al. Self-report compared to electronic medical record across eight adult vaccines: do results vary by
April 2015
391
demographic factors? Vaccine 2013;31(37):3928–35. http://dx.doi.org/ 10.1016/j.vaccine.2013.06.041. 18. Lindley MC, Wortley PM, Winston CA, Bardenheier BH. The role of attitudes in understanding disparities in adult influenza vaccination. Am J Prev Med. 2006;31(4):281–5. http://dx.doi.org/10.1016/j.amepre.2006.06.025. 19. Lochner KA, Wynne M. Flu shots and the characteristics of unvaccinated elderly Medicare beneficiaries. Medicare Medicaid Res Rev 2011;1(4):E1–E11. http://dx.doi.org/10.5600/mmrr.001.04.b01. 20. Wooten KG, Wortley PM, Singleton JA, Euler GL. Perceptions matter: beliefs about influenza vaccine and vaccination behavior among elderly white, black and Hispanic Americans. Vaccine 2012;30(48):6927–34. http://dx.doi.org/10.1016/j.vaccine.2012.08.036.
