Journal of Adolescent Health xxx (2014) 1e7
www.jahonline.org Original article
Measuring Youth Health Engagement: Development of the Youth Engagement With Health Services Survey Rachel A. Sebastian, M.A. a, *, Mary M. Ramos, M.D., M.P.H. b, Scott Stumbo, M.A. c, Jane McGrath, M.D. b, and Gerry Fairbrother, Ph.D. d a
Child Policy & Population Health, James M. Anderson Center for Health Systems Excellence, Cincinnati Children’s Hospital Medical Center, Cincinnati, Ohio Department of Pediatrics, University of New Mexico, Albuquerque, New Mexico c Child and Adolescent Health Measurement Initiative, Oregon Health and Science University, Portland, Oregon d Academy Health, Washington, DC b
Article history: Received October 23, 2013; Accepted February 11, 2014 Keywords: Adolescent health services; Quality of health care; Patient engagement; Self-efficacy; Health literacy; Preventive health services
A B S T R A C T
Purpose: The purpose of this study was to create and validate a survey instrument designed to measure Youth Engagement with Health Services (YEHS!). Methods: A 61-item YEHS! survey was created through a multistaged process, which included literature review, subject matter expert opinion, review of existing validated measures, and cognitive interviewing with 41 adolescents in Colorado and New Mexico. The YEHS! was then pilot tested with a diverse group of high school students (n ¼ 354) accessing health services at one of eight school-based health centers in Colorado and New Mexico. We conducted psychometric analyses and examined correlations between the youth health engagement scales and measures of quality of care. Results: We created scales to measure two domains of youth health engagement: health access literacy and health self-efficacy. The youth health engagement scales demonstrated strong reliability (Cronbach’s a .76 and .82) and construct validity (mean factor loading .71 and .76). Youth health engagement scores predicted higher experiences of care scores (p < .001) and receipt of more anticipatory guidance (p < .01). Conclusions: This study supports the YEHS! as a valid and reliable measure of youth health engagement among adolescents using school-based health centers. We demonstrate an association between youth health engagement and two quality of care measures. Additional testing is needed to ensure the reliability and validity of the instrument in diverse adolescent populations. Ó 2014 Society for Adolescent Health and Medicine. All rights reserved.
Changes in the health care delivery system, including an emphasis on cost-effective and high-quality outpatient care and patient-centered options for the treatment of chronic conditions, have resulted in increasing expectations that patients be engaged
Conflicts of Interest: The authors have no conflicts of interest to disclose. * Address correspondence to: Rachel A. Sebastian, M.A., Cincinnati Children’s Hospital Medical Center, James M. Anderson Center for Health Systems Excellence, 3333 Burnet Avenue, MLC 7014, Cincinnati, OH 45229-3039. E-mail address: [email protected] (R.A. Sebastian).
IMPLICATIONS AND CONTRIBUTION
This article presents the development and validation of a new survey instrument that measures youth health engagement. We report the association between youth health engagement and measures of adolescent health care quality. This survey may be useful in studies evaluating adolescent health care experiences.
with their own health care [1e4]. Patients who are more involved in their care are more satisfied with the care they receive and have better outcomes [3,5e7]. Patients with more health knowledge (health literacy) and who are involved in the decision-making about their treatment options are more likely to adhere to medication and treatment plans, resulting in better management of health conditions [1,7e9]. However, the field of patient engagement is relatively new, and research to date has centered on adults, more specifically those with chronic conditions [5,7,9,10]. Even for adults,
1054-139X/$ e see front matter Ó 2014 Society for Adolescent Health and Medicine. All rights reserved. http://dx.doi.org/10.1016/j.jadohealth.2014.02.008
2
R.A. Sebastian et al. / Journal of Adolescent Health xxx (2014) 1e7
relatively few studies exist dealing with key aspects of engagement, including its conceptualization and measurement [1,2,11,12], implementation of engagement strategies [7,9,13e15], and the connection between engagement and health outcomes [3,5,8,16,17]. There is a growing interest in expanding these concepts to youth, including provision of youth-friendly services [18e20] and engaging youth productively in their own care [10,21e23]. To support this direction in the provision of services to youth, we need a tool to measure youth engagement with their health care. To fill this gap, we developed a scale to measure levels of engagement of adolescents with their health care. This scale is included in a survey that also includes measures of health care utilization, experiences with care, health risks, and receipt of anticipatory guidance. The intent was that the resulting instrument could be used not only to assess engagement with services but also to relate this engagement to other aspects of care. This was done as part of the school-based health center (SBHC) improvement project in Colorado and New Mexico, which is 1 of the 10 national demonstration projects funded through the Children’s Health Insurance Program Reauthorization [24]. The purpose of this article is to describe the development of the youth health engagement scale. The results indicate that the items developed to measure youth health engagement are valid and reliable measures of this construct within the adolescent populations in which it was tested.
A similar construct that has been discussed in the literature is that of transition readiness, which focuses on the ability of adolescents with special health care needs to effectively make the transition from pediatric to adult care and from care managed by a parent or caregiver to management of their own care [25,26]. Measures of self-efficacy and disease-specific literacy have been included in tools developed to measure transition readiness [25,26]. However, similar to other tools measuring aspects of engagement, these tools have been developed for patients with chronic conditions or special health care needs. Presumably, adolescents without special health care needs also struggle with the transition from pediatric to adult care, and many of the same skills and knowledge to effectively do so are required. Based on the components of patient engagement found in the literature, we conceptualized youth health engagement as being composed of health access literacy (knowledge) and health selfefficacy (incorporating elements of self-advocacy, effective communication with providers, and ability to follow through on plans made with providers). We developed items intended to measure these two components of youth health engagement and hypothesized that these components would correlate with one another, as measures of one underlying construct (youth health engagement), and with measures of quality of care, including adolescents’ ratings of their experience of care and reported receipt of anticipatory guidance. Instrument development
Methods Ethical review The study was approved by the Human Research Protections Office of The University of New Mexico Health Sciences Center and Cincinnati Children’s Hospital Medical Center. Conceptualizing patient engagement for adolescents Patient engagement has been defined as “actions individuals must take to obtain the greatest benefit from the health care services available to them” and focuses on the behaviors of individuals, rather than the actions of professionals or policies of institutions [1]. The most well-known measurement of patient engagement is Hibbard’s patient activation measure [12], which identifies and measures four components of patient activation: locus of control (or the belief that the patient is responsible for and can affect their own health); knowledge (of health and health problems); confidence (in the patient’s own ability to manage their health, interact effectively with providers, and adhere to treatment plans); and action (the extent to which the patient is adhering to treatment plans). These four components were later used as the basis for Gruman et al.’s [1] comprehensive list of engagement behaviors. Central to both of these conceptualizations of engagement are elements of seeking health care and health information, working in partnership with providers, managing one’s own health care (i.e., keeping appointments and making lists of questions), and health-promoting behaviors (including treatment adherence, preventive care, and adoption of healthy habits). Measuring the health engagement of adolescents requires a conceptualization of the health-engaged young person, which takes into account adolescents’ developmental stage and recognizes adolescents’ growing knowledge base and emerging abilities to communicate and advocate for their own needs.
The Youth Engagement with Health Services (YEHS!) survey instrument includes several components, including the youth health engagement scale as well as measures of experiences with health care, reported receipt of anticipatory guidance, and demographic information. To measure experiences of care, we included five items from the Experiences of Care Scale, developed for the Consumer Assessment of Healthcare Providers and Systems survey [27]. To measure the receipt of recommended anticipatory guidance, we created survey items that were based (with permission) on the American Academy of Pediatrics Bright Futures Guidelines and the Young Adult Health Care Survey [28,29]. We included four topical domains from Bright Futures Guidelines: physical growth and development; academic and social competence; emotional well-being; and sexual health risk reduction, as these domains corresponded to our Children’s Health Insurance Program Reauthorization grant improvement areas [24]. Developing the youth health engagement scale items for the YEHS! survey involved multiple steps to operationalize the constructs, including a review of existing literature on health literacy, health self-efficacy, and patient engagement; subject matter expert opinion; focus groups with youth to assess face and content validity; and analysis of data from the pilot administration of the survey. Survey items measuring health access literacy include items related to knowledge of where to get care, health insurance, and confidential services. Items measuring health self-efficacy include those related to accessing care, working in partnership with health care providers, and carrying out plans made with a provider. Cognitive testing In the winter of 2012, a draft of the survey was administered with five groups of adolescents in Colorado and New Mexico. All
R.A. Sebastian et al. / Journal of Adolescent Health xxx (2014) 1e7
but one of these groups were SBHC users. The focus groups included a total of 41 adolescents, aged 14e18 years. The adolescents took the survey individually and then discussed specific questions and concepts from the survey in focus-group format. The adolescents also completed a separate questionnaire during the focus groups that included follow-up questions to several of the survey items. Data from the completed surveys and followup questionnaires were analyzed to assess the face and content validity of the survey items. Participants were provided written and verbal information about the purposes of the study and given $30 as compensation for their time. Informed assent was attained from the adolescents participating; however, to protect confidentiality, written documentation of assent and parental permission were both waived. Survey administration The YEHS! survey was administered April 2012 through June 2012 in eight SBHCs in Colorado and New Mexico participating in the SBHC improvement project. The survey was administered on a tablet by SBHC staff members. Quality improvement coaches working with the sites provided guidance on survey administration, including recruitment, ensuring eligibility (only high school students who were patients of the SBHC and who had a visit within the past year were eligible to take the survey), providing incentives, and tracking survey completion. Students who completed the survey received $5 gift cards as compensation for their time. The survey is anonymous and includes no identifying information. Students were given a private place to take the survey and were informed that their providers would not be reviewing their individual survey results. Written documentation of informed consent and parental permission were both waived in the interest of protecting the privacy of students receiving services at SBHCs, some of whom are receiving confidential services. The survey data were not connected to the students’ medical records, and only aggregate reports were provided to the SBHC staff. Data analysis We analyzed data from the pilot administration to assess the reliability and validity of the youth health engagement subscales. We conducted psychometric analyses, including Cronbach’s as and exploratory factor analysis, to assess the reliability of the health access literacy and health self-efficacy scales. To assess validity of the scales, we performed bivariate correlations and ordinary least squares regression analyses to look for expected correlations with the experience of care measure and the receipt of anticipatory guidance. Results Characteristics of sample from pilot administration The pilot administration yielded 354 completed surveys from a diverse population of high school students using SBHCs in Colorado and New Mexico (Table 1). The mean age of the respondents was 16 years, and the majority were female and Hispanic, consistent with the demographics of the patient populations at these SBHCs. The majority of respondents reported having a usual source of care and having had a private visit with a provider within the past year.
3
Table 2 lists the results of the anticipatory guidance and experience of care items. Respondents reported having received guidance within the past 12 months on many topics. The topic area with the highest reported receipt of guidance was sexual health risk reduction. Emotional well-being had the lowest reported receipt of guidance. Overall, respondents rated their experiences with health care highly, with a mean score of 3.32 (with 4 being the highest) on the experience of care scale (Table 2). Youth health engagement scales Table 3 lists the reliability and factor analysis results for the youth health engagement subscales. For each item, respondents could respond “Strongly Disagree,” “Somewhat Disagree,” “Somewhat Agree,” or “Strongly Agree.” The responses were scored 1 through 4, where 1 equals “Strongly Disagree” and 4 equals “Strongly Agree.” The response categories for the health access literacy items included a “Don’t know/Not sure” response option under the assumption that some students would not have any health insurance and would not be able to respond to the items about health insurance. Prior to analysis, the “Don’t know/ Not sure” responses were recoded to missing, so as not to influence the overall scale mean. Health access literacy subscale The initial analysis of all seven items piloted in the health access literacy section resulted in a Cronbach’s a of .831. However, given concerns about the possibility of redundancy among the three health insurance items, which were strongly correlated with one another, two of these items were removed from the final scale (Table 3). The final health access literacy scale includes Table 1 Demographic and selected characteristics of respondents of pilot survey of YEHS!
Age group, years 11e14 15e17 18þ Sex Female Race/ethnicity Hispanic White, non-Hispanic American-Indian/Alaska Native, non-Hispanic Black, Asian, or multiracial, non-Hispanic LGBTQ Risk behaviors At risk for depression in past yeara Ever had sex Health care utilization Has a “usual” source of care (excluding emergency room/urgent care use as usual source of care) Had a preventive care visit within past year Health communications Within past year, had a private visit with a provider Within past year, a provider told them what was discussed was confidential
No.
%
55 228 62
15.9 66.1 18.0
226
68.9
215 64 50 15 24
62.5 18.6 14.5 4.4 7.0
101 207
29.2 59.8
309
89.3
278
80.3
253 256
74.6 75.1
N ¼ 354 respondents to the YEHS! survey. LGBTQ ¼ lesbian, gay, bisexual, transgender, or questioning. Respondents were asked “Are you, or do you ever wonder if you are lesbian, gay, bisexual, or transgender?”; YEHS! ¼ Youth Engagement with Health Services. a At risk for depression in past year. Respondents were asked “During the last 12 months, did you ever feel so sad or hopeless for two weeks or more in a row that you stopped doing some usual activities?”
4
R.A. Sebastian et al. / Journal of Adolescent Health xxx (2014) 1e7
Table 2 Receipt of anticipatory guidance and experience of care measures A. Anticipatory guidance topics Physical growth and development Oral care (brushing, flossing, and seeing a dentist) Weight Healthy eating or diet Physical activity or exercise Body image or appearance Social and academic competence Your family Your friends Your school performance or grades Your future plans (after high school) Emotional well-being Your emotions or moods Suicide How you deal with stress Sleep Sexual orientation or gender identity Sexual risk reduction STDs (such as gonorrhea or chlamydia) Condoms Choosing not to have sex Not having sex to prevent STDs and pregnancy Birth control
Percent of respondentsa 58.9 50.0 49.1 52.5 36.4 48.1 37.6 39.7 53.0
The analysis indicated that the item with the least amount of correlation with the others was the item addressing carrying an insurance card. This item generally scored lower than the others in the subscale, and given concerns about the face validity of the item (that many of the students responding to the survey do not have health insurance and thus would not have a card to carry), this item was dropped, resulting in a five-item scale with a mean of 3.11 and a Cronbach’s a of .817. An exploratory factor analysis of these five items results in all five items loading strongly onto one underlying factor. These results suggest that the items are strongly correlated and are likely to be measuring one underlying construct (Table 3). Correlations between subscales
46.8 26.1 43.8 41.1 28.1 60.4 63.2 51.5 54.0 60.9
Health access literacy and health self-efficacy were hypothesized to be subscales of the larger concept of youth health engagement and as such were expected to be positively correlated with one another. A bivariate analysis of the two subscales was performed, and the results indicate that they are correlated such that higher scores on the health access literacy subscale are associated with higher scores on the health self-efficacy subscale (r ¼ .211, p < .001).
B. Experience of care scale
Percent of respondentsb
Correlations with experience of care scores
In the last 12 months, how often did doctors or other health care providers listen carefully to you? In the last 12 months, how often did you have a hard time speaking with or understanding your doctor or other health care provider because you spoke different languages? In the last 12 months, how often did doctors or other health care providers explain things in a way that you could understand? In the last 12 months, how often did doctors or other health care providers show respect for what you had to say? In the last 12 months, how often did doctors or other health care providers spend enough time with you? Mean experiences of care score (on a scale of 1e4, with 4 being the highest)
86.6
19.1
80.5
87.6
79.1
3.32 (SD ¼ .61)
SD ¼ standard deviation; STD ¼ sexually transmitted disease. a Percent who said they received anticipatory guidance on these items within the past 12 months. b Percent who responded “usually” or “always” to statements about their experiences with health care providers within the past 12 months.
only one item addressing knowledge of health insuranced knowing how to use health insurance to obtain services and medications. The final health access literacy scale includes five items with a mean of 3.11 and a Cronbach’s a of .758. While the five-item scale resulted in a lower Cronbach’s a than the sevenitem scale, this is thought to be a more accurate reflection of the correlation among these different yet related items. When two of the health insurance items are removed, all five items load strongly onto one factor, indicating that all five items are measuring one underlying construct. Health self-efficacy subscale To assess the reliability of the health self-efficacy subscale, all six items were analyzed as a single subscale, producing an a of .778.
Table 4 lists the results of an ordinary least squares regression model of the relationships among health access literacy, health self-efficacy, and the combined youth health engagement scale with experience of care scores. Values are shown with and without adjustment for age, sex, and race/ethnicity. All three constructs (health access literacy, health self-efficacy, and youth health engagement) are positively associated with experience of care. These associations change little when adjusting the model for age, sex, and race/ethnicity (Table 4). Correlations with receipt of anticipatory guidance Table 4 also lists the results of an ordinary least squares regression model of the relationships among health access literacy, health self-efficacy, and the combined youth health engagement scale with receipt of recommended anticipatory guidance, with and without adjustment for age, sex, and race/ ethnicity. Health access literacy, health self-efficacy, and youth health engagement are all positively associated with the receipt of recommended anticipatory guidance even when adjusting the model for age, sex, and race/ethnicity. Discussion In this study, we developed a youth-oriented patient engagement scale included in a survey measuring health risks, receipt of recommended anticipatory guidance, and experiences with care to create a new instrument for adolescentsdthe YEHS! survey. We conceptualized engagement of adolescents with their health care as having two components: (1) having requisite knowledge to access health care and (2) having the ability to interact effectively with the health care system and their providers. These components translated into two subscales: health access literacy and health self-efficacy. We developed and tested these subscales through a rigorous process that included cognitive testing with youth, as well as input from a national advisory panel of experts in
R.A. Sebastian et al. / Journal of Adolescent Health xxx (2014) 1e7
5
Table 3 Youth health engagement subscalesdhealth access literacy and health self-efficacy, N ¼ 354
Health access literacy I know where to get medical care when the SBHC is closed There are adults I can talk to about questions or concerns I have about my health I know how to use my health insurance to get health care or medication I am confident that I understand which services are confidential and which are not I know of a place (other than the SBHC) where teenagers can go to see a doctor or other health care provider without their parents/guardians knowing about it Health self-efficacy I will tell a doctor or other health care provider my concerns, even if they do not ask I talk to my doctor or other health care provider about different options to address health problems or concerns I make appointments for myself to see a doctor or other health care provider When I make a plan with a doctor or other health care provider, I can follow through on the plan at home I have a safe and trusting relationship with at least one doctor or other health care provider
SD
Cronbach’s a
Factor loading
.758
.712 .754 .647
% Strongly/somewhat agree
Mean
90.1 89.8
3.11 3.41 3.53
66.1
2.86
75.9
3.07
.96
.736
70.8
3.00
1.14
.679
81.0
3.11 3.09
.68 .89
84.1
3.12
.83
.828
68.4
2.91
1.03
.743
82.9
3.16
.82
.694
82.3
3.23
.88
.771
.95 .76 .75 1.1
.746
.817
.76 .764
Omitted from final scale were four items: “I know what kind of insurance I have” and “I know what health care services I can get using my health insurance” were omitted from Health Access Literacy subscale during factor analysis. “I have a health insurance card that I carry with me” was omitted from health self-efficacy subscale during factor analysis. One item “I know how to make an appointment with a health care provider” was omitted during cognitive testing. SD ¼ standard deviation.
adolescent health that included adolescents. Results indicated that these two subscales are reliable and valid measures of youth health engagement. The five health access literacy items and five self-health efficacy items each demonstrate high internal consistency (Cronbach’s a of .758 and .817, respectively). Further, exploratory factor analysis results showed that in both cases, the five items loaded onto a single factor. The two subscales were positively correlated, indicating that they were both measuring the same underlying concept. Regression analyses showing high association between the engagement subscales and positive experiences of care and receipt of recommended anticipatory guidance indicated that the scales not only were reliable but also had high construct validity. This is the first study to develop and establish reliability and validity for a patient engagement instrument appropriate for adolescents. While a number of existing instruments measure patient and family experiences of health care [2,27,29,30], instruments measuring patient engagement have focused on adults, particularly chronically ill adults [12,15,31,32]. The YEHS! instrument, in contrast, seeks to measure engagement of adolescents, including nonchronically ill adolescents, using items that are developmentally appropriate and that address engagement with aspects of care important to youth. While our conceptualization of youth health engagement is consistent with that of patient engagement among adults, the youth health engagement scale addresses elements of engagement that are particularly pertinent to adolescents. For example, knowledge of confidential services is important for adolescents given their increasing need to access health care on their own. Further, it addresses elements of engagement that will ensure that they develop into adults who can effectively maneuver through the health care system (i.e., knowing where to get care and knowing how to use health insurance).
Adolescent health care and adolescent perspectives on health care have historically received little attention. From 2010 to 2012,
www.jahonline.org Original article
Measuring Youth Health Engagement: Development of the Youth Engagement With Health Services Survey Rachel A. Sebastian, M.A. a, *, Mary M. Ramos, M.D., M.P.H. b, Scott Stumbo, M.A. c, Jane McGrath, M.D. b, and Gerry Fairbrother, Ph.D. d a
Child Policy & Population Health, James M. Anderson Center for Health Systems Excellence, Cincinnati Children’s Hospital Medical Center, Cincinnati, Ohio Department of Pediatrics, University of New Mexico, Albuquerque, New Mexico c Child and Adolescent Health Measurement Initiative, Oregon Health and Science University, Portland, Oregon d Academy Health, Washington, DC b
Article history: Received October 23, 2013; Accepted February 11, 2014 Keywords: Adolescent health services; Quality of health care; Patient engagement; Self-efficacy; Health literacy; Preventive health services
A B S T R A C T
Purpose: The purpose of this study was to create and validate a survey instrument designed to measure Youth Engagement with Health Services (YEHS!). Methods: A 61-item YEHS! survey was created through a multistaged process, which included literature review, subject matter expert opinion, review of existing validated measures, and cognitive interviewing with 41 adolescents in Colorado and New Mexico. The YEHS! was then pilot tested with a diverse group of high school students (n ¼ 354) accessing health services at one of eight school-based health centers in Colorado and New Mexico. We conducted psychometric analyses and examined correlations between the youth health engagement scales and measures of quality of care. Results: We created scales to measure two domains of youth health engagement: health access literacy and health self-efficacy. The youth health engagement scales demonstrated strong reliability (Cronbach’s a .76 and .82) and construct validity (mean factor loading .71 and .76). Youth health engagement scores predicted higher experiences of care scores (p < .001) and receipt of more anticipatory guidance (p < .01). Conclusions: This study supports the YEHS! as a valid and reliable measure of youth health engagement among adolescents using school-based health centers. We demonstrate an association between youth health engagement and two quality of care measures. Additional testing is needed to ensure the reliability and validity of the instrument in diverse adolescent populations. Ó 2014 Society for Adolescent Health and Medicine. All rights reserved.
Changes in the health care delivery system, including an emphasis on cost-effective and high-quality outpatient care and patient-centered options for the treatment of chronic conditions, have resulted in increasing expectations that patients be engaged
Conflicts of Interest: The authors have no conflicts of interest to disclose. * Address correspondence to: Rachel A. Sebastian, M.A., Cincinnati Children’s Hospital Medical Center, James M. Anderson Center for Health Systems Excellence, 3333 Burnet Avenue, MLC 7014, Cincinnati, OH 45229-3039. E-mail address: [email protected] (R.A. Sebastian).
IMPLICATIONS AND CONTRIBUTION
This article presents the development and validation of a new survey instrument that measures youth health engagement. We report the association between youth health engagement and measures of adolescent health care quality. This survey may be useful in studies evaluating adolescent health care experiences.
with their own health care [1e4]. Patients who are more involved in their care are more satisfied with the care they receive and have better outcomes [3,5e7]. Patients with more health knowledge (health literacy) and who are involved in the decision-making about their treatment options are more likely to adhere to medication and treatment plans, resulting in better management of health conditions [1,7e9]. However, the field of patient engagement is relatively new, and research to date has centered on adults, more specifically those with chronic conditions [5,7,9,10]. Even for adults,
1054-139X/$ e see front matter Ó 2014 Society for Adolescent Health and Medicine. All rights reserved. http://dx.doi.org/10.1016/j.jadohealth.2014.02.008
2
R.A. Sebastian et al. / Journal of Adolescent Health xxx (2014) 1e7
relatively few studies exist dealing with key aspects of engagement, including its conceptualization and measurement [1,2,11,12], implementation of engagement strategies [7,9,13e15], and the connection between engagement and health outcomes [3,5,8,16,17]. There is a growing interest in expanding these concepts to youth, including provision of youth-friendly services [18e20] and engaging youth productively in their own care [10,21e23]. To support this direction in the provision of services to youth, we need a tool to measure youth engagement with their health care. To fill this gap, we developed a scale to measure levels of engagement of adolescents with their health care. This scale is included in a survey that also includes measures of health care utilization, experiences with care, health risks, and receipt of anticipatory guidance. The intent was that the resulting instrument could be used not only to assess engagement with services but also to relate this engagement to other aspects of care. This was done as part of the school-based health center (SBHC) improvement project in Colorado and New Mexico, which is 1 of the 10 national demonstration projects funded through the Children’s Health Insurance Program Reauthorization [24]. The purpose of this article is to describe the development of the youth health engagement scale. The results indicate that the items developed to measure youth health engagement are valid and reliable measures of this construct within the adolescent populations in which it was tested.
A similar construct that has been discussed in the literature is that of transition readiness, which focuses on the ability of adolescents with special health care needs to effectively make the transition from pediatric to adult care and from care managed by a parent or caregiver to management of their own care [25,26]. Measures of self-efficacy and disease-specific literacy have been included in tools developed to measure transition readiness [25,26]. However, similar to other tools measuring aspects of engagement, these tools have been developed for patients with chronic conditions or special health care needs. Presumably, adolescents without special health care needs also struggle with the transition from pediatric to adult care, and many of the same skills and knowledge to effectively do so are required. Based on the components of patient engagement found in the literature, we conceptualized youth health engagement as being composed of health access literacy (knowledge) and health selfefficacy (incorporating elements of self-advocacy, effective communication with providers, and ability to follow through on plans made with providers). We developed items intended to measure these two components of youth health engagement and hypothesized that these components would correlate with one another, as measures of one underlying construct (youth health engagement), and with measures of quality of care, including adolescents’ ratings of their experience of care and reported receipt of anticipatory guidance. Instrument development
Methods Ethical review The study was approved by the Human Research Protections Office of The University of New Mexico Health Sciences Center and Cincinnati Children’s Hospital Medical Center. Conceptualizing patient engagement for adolescents Patient engagement has been defined as “actions individuals must take to obtain the greatest benefit from the health care services available to them” and focuses on the behaviors of individuals, rather than the actions of professionals or policies of institutions [1]. The most well-known measurement of patient engagement is Hibbard’s patient activation measure [12], which identifies and measures four components of patient activation: locus of control (or the belief that the patient is responsible for and can affect their own health); knowledge (of health and health problems); confidence (in the patient’s own ability to manage their health, interact effectively with providers, and adhere to treatment plans); and action (the extent to which the patient is adhering to treatment plans). These four components were later used as the basis for Gruman et al.’s [1] comprehensive list of engagement behaviors. Central to both of these conceptualizations of engagement are elements of seeking health care and health information, working in partnership with providers, managing one’s own health care (i.e., keeping appointments and making lists of questions), and health-promoting behaviors (including treatment adherence, preventive care, and adoption of healthy habits). Measuring the health engagement of adolescents requires a conceptualization of the health-engaged young person, which takes into account adolescents’ developmental stage and recognizes adolescents’ growing knowledge base and emerging abilities to communicate and advocate for their own needs.
The Youth Engagement with Health Services (YEHS!) survey instrument includes several components, including the youth health engagement scale as well as measures of experiences with health care, reported receipt of anticipatory guidance, and demographic information. To measure experiences of care, we included five items from the Experiences of Care Scale, developed for the Consumer Assessment of Healthcare Providers and Systems survey [27]. To measure the receipt of recommended anticipatory guidance, we created survey items that were based (with permission) on the American Academy of Pediatrics Bright Futures Guidelines and the Young Adult Health Care Survey [28,29]. We included four topical domains from Bright Futures Guidelines: physical growth and development; academic and social competence; emotional well-being; and sexual health risk reduction, as these domains corresponded to our Children’s Health Insurance Program Reauthorization grant improvement areas [24]. Developing the youth health engagement scale items for the YEHS! survey involved multiple steps to operationalize the constructs, including a review of existing literature on health literacy, health self-efficacy, and patient engagement; subject matter expert opinion; focus groups with youth to assess face and content validity; and analysis of data from the pilot administration of the survey. Survey items measuring health access literacy include items related to knowledge of where to get care, health insurance, and confidential services. Items measuring health self-efficacy include those related to accessing care, working in partnership with health care providers, and carrying out plans made with a provider. Cognitive testing In the winter of 2012, a draft of the survey was administered with five groups of adolescents in Colorado and New Mexico. All
R.A. Sebastian et al. / Journal of Adolescent Health xxx (2014) 1e7
but one of these groups were SBHC users. The focus groups included a total of 41 adolescents, aged 14e18 years. The adolescents took the survey individually and then discussed specific questions and concepts from the survey in focus-group format. The adolescents also completed a separate questionnaire during the focus groups that included follow-up questions to several of the survey items. Data from the completed surveys and followup questionnaires were analyzed to assess the face and content validity of the survey items. Participants were provided written and verbal information about the purposes of the study and given $30 as compensation for their time. Informed assent was attained from the adolescents participating; however, to protect confidentiality, written documentation of assent and parental permission were both waived. Survey administration The YEHS! survey was administered April 2012 through June 2012 in eight SBHCs in Colorado and New Mexico participating in the SBHC improvement project. The survey was administered on a tablet by SBHC staff members. Quality improvement coaches working with the sites provided guidance on survey administration, including recruitment, ensuring eligibility (only high school students who were patients of the SBHC and who had a visit within the past year were eligible to take the survey), providing incentives, and tracking survey completion. Students who completed the survey received $5 gift cards as compensation for their time. The survey is anonymous and includes no identifying information. Students were given a private place to take the survey and were informed that their providers would not be reviewing their individual survey results. Written documentation of informed consent and parental permission were both waived in the interest of protecting the privacy of students receiving services at SBHCs, some of whom are receiving confidential services. The survey data were not connected to the students’ medical records, and only aggregate reports were provided to the SBHC staff. Data analysis We analyzed data from the pilot administration to assess the reliability and validity of the youth health engagement subscales. We conducted psychometric analyses, including Cronbach’s as and exploratory factor analysis, to assess the reliability of the health access literacy and health self-efficacy scales. To assess validity of the scales, we performed bivariate correlations and ordinary least squares regression analyses to look for expected correlations with the experience of care measure and the receipt of anticipatory guidance. Results Characteristics of sample from pilot administration The pilot administration yielded 354 completed surveys from a diverse population of high school students using SBHCs in Colorado and New Mexico (Table 1). The mean age of the respondents was 16 years, and the majority were female and Hispanic, consistent with the demographics of the patient populations at these SBHCs. The majority of respondents reported having a usual source of care and having had a private visit with a provider within the past year.
3
Table 2 lists the results of the anticipatory guidance and experience of care items. Respondents reported having received guidance within the past 12 months on many topics. The topic area with the highest reported receipt of guidance was sexual health risk reduction. Emotional well-being had the lowest reported receipt of guidance. Overall, respondents rated their experiences with health care highly, with a mean score of 3.32 (with 4 being the highest) on the experience of care scale (Table 2). Youth health engagement scales Table 3 lists the reliability and factor analysis results for the youth health engagement subscales. For each item, respondents could respond “Strongly Disagree,” “Somewhat Disagree,” “Somewhat Agree,” or “Strongly Agree.” The responses were scored 1 through 4, where 1 equals “Strongly Disagree” and 4 equals “Strongly Agree.” The response categories for the health access literacy items included a “Don’t know/Not sure” response option under the assumption that some students would not have any health insurance and would not be able to respond to the items about health insurance. Prior to analysis, the “Don’t know/ Not sure” responses were recoded to missing, so as not to influence the overall scale mean. Health access literacy subscale The initial analysis of all seven items piloted in the health access literacy section resulted in a Cronbach’s a of .831. However, given concerns about the possibility of redundancy among the three health insurance items, which were strongly correlated with one another, two of these items were removed from the final scale (Table 3). The final health access literacy scale includes Table 1 Demographic and selected characteristics of respondents of pilot survey of YEHS!
Age group, years 11e14 15e17 18þ Sex Female Race/ethnicity Hispanic White, non-Hispanic American-Indian/Alaska Native, non-Hispanic Black, Asian, or multiracial, non-Hispanic LGBTQ Risk behaviors At risk for depression in past yeara Ever had sex Health care utilization Has a “usual” source of care (excluding emergency room/urgent care use as usual source of care) Had a preventive care visit within past year Health communications Within past year, had a private visit with a provider Within past year, a provider told them what was discussed was confidential
No.
%
55 228 62
15.9 66.1 18.0
226
68.9
215 64 50 15 24
62.5 18.6 14.5 4.4 7.0
101 207
29.2 59.8
309
89.3
278
80.3
253 256
74.6 75.1
N ¼ 354 respondents to the YEHS! survey. LGBTQ ¼ lesbian, gay, bisexual, transgender, or questioning. Respondents were asked “Are you, or do you ever wonder if you are lesbian, gay, bisexual, or transgender?”; YEHS! ¼ Youth Engagement with Health Services. a At risk for depression in past year. Respondents were asked “During the last 12 months, did you ever feel so sad or hopeless for two weeks or more in a row that you stopped doing some usual activities?”
4
R.A. Sebastian et al. / Journal of Adolescent Health xxx (2014) 1e7
Table 2 Receipt of anticipatory guidance and experience of care measures A. Anticipatory guidance topics Physical growth and development Oral care (brushing, flossing, and seeing a dentist) Weight Healthy eating or diet Physical activity or exercise Body image or appearance Social and academic competence Your family Your friends Your school performance or grades Your future plans (after high school) Emotional well-being Your emotions or moods Suicide How you deal with stress Sleep Sexual orientation or gender identity Sexual risk reduction STDs (such as gonorrhea or chlamydia) Condoms Choosing not to have sex Not having sex to prevent STDs and pregnancy Birth control
Percent of respondentsa 58.9 50.0 49.1 52.5 36.4 48.1 37.6 39.7 53.0
The analysis indicated that the item with the least amount of correlation with the others was the item addressing carrying an insurance card. This item generally scored lower than the others in the subscale, and given concerns about the face validity of the item (that many of the students responding to the survey do not have health insurance and thus would not have a card to carry), this item was dropped, resulting in a five-item scale with a mean of 3.11 and a Cronbach’s a of .817. An exploratory factor analysis of these five items results in all five items loading strongly onto one underlying factor. These results suggest that the items are strongly correlated and are likely to be measuring one underlying construct (Table 3). Correlations between subscales
46.8 26.1 43.8 41.1 28.1 60.4 63.2 51.5 54.0 60.9
Health access literacy and health self-efficacy were hypothesized to be subscales of the larger concept of youth health engagement and as such were expected to be positively correlated with one another. A bivariate analysis of the two subscales was performed, and the results indicate that they are correlated such that higher scores on the health access literacy subscale are associated with higher scores on the health self-efficacy subscale (r ¼ .211, p < .001).
B. Experience of care scale
Percent of respondentsb
Correlations with experience of care scores
In the last 12 months, how often did doctors or other health care providers listen carefully to you? In the last 12 months, how often did you have a hard time speaking with or understanding your doctor or other health care provider because you spoke different languages? In the last 12 months, how often did doctors or other health care providers explain things in a way that you could understand? In the last 12 months, how often did doctors or other health care providers show respect for what you had to say? In the last 12 months, how often did doctors or other health care providers spend enough time with you? Mean experiences of care score (on a scale of 1e4, with 4 being the highest)
86.6
19.1
80.5
87.6
79.1
3.32 (SD ¼ .61)
SD ¼ standard deviation; STD ¼ sexually transmitted disease. a Percent who said they received anticipatory guidance on these items within the past 12 months. b Percent who responded “usually” or “always” to statements about their experiences with health care providers within the past 12 months.
only one item addressing knowledge of health insuranced knowing how to use health insurance to obtain services and medications. The final health access literacy scale includes five items with a mean of 3.11 and a Cronbach’s a of .758. While the five-item scale resulted in a lower Cronbach’s a than the sevenitem scale, this is thought to be a more accurate reflection of the correlation among these different yet related items. When two of the health insurance items are removed, all five items load strongly onto one factor, indicating that all five items are measuring one underlying construct. Health self-efficacy subscale To assess the reliability of the health self-efficacy subscale, all six items were analyzed as a single subscale, producing an a of .778.
Table 4 lists the results of an ordinary least squares regression model of the relationships among health access literacy, health self-efficacy, and the combined youth health engagement scale with experience of care scores. Values are shown with and without adjustment for age, sex, and race/ethnicity. All three constructs (health access literacy, health self-efficacy, and youth health engagement) are positively associated with experience of care. These associations change little when adjusting the model for age, sex, and race/ethnicity (Table 4). Correlations with receipt of anticipatory guidance Table 4 also lists the results of an ordinary least squares regression model of the relationships among health access literacy, health self-efficacy, and the combined youth health engagement scale with receipt of recommended anticipatory guidance, with and without adjustment for age, sex, and race/ ethnicity. Health access literacy, health self-efficacy, and youth health engagement are all positively associated with the receipt of recommended anticipatory guidance even when adjusting the model for age, sex, and race/ethnicity. Discussion In this study, we developed a youth-oriented patient engagement scale included in a survey measuring health risks, receipt of recommended anticipatory guidance, and experiences with care to create a new instrument for adolescentsdthe YEHS! survey. We conceptualized engagement of adolescents with their health care as having two components: (1) having requisite knowledge to access health care and (2) having the ability to interact effectively with the health care system and their providers. These components translated into two subscales: health access literacy and health self-efficacy. We developed and tested these subscales through a rigorous process that included cognitive testing with youth, as well as input from a national advisory panel of experts in
R.A. Sebastian et al. / Journal of Adolescent Health xxx (2014) 1e7
5
Table 3 Youth health engagement subscalesdhealth access literacy and health self-efficacy, N ¼ 354
Health access literacy I know where to get medical care when the SBHC is closed There are adults I can talk to about questions or concerns I have about my health I know how to use my health insurance to get health care or medication I am confident that I understand which services are confidential and which are not I know of a place (other than the SBHC) where teenagers can go to see a doctor or other health care provider without their parents/guardians knowing about it Health self-efficacy I will tell a doctor or other health care provider my concerns, even if they do not ask I talk to my doctor or other health care provider about different options to address health problems or concerns I make appointments for myself to see a doctor or other health care provider When I make a plan with a doctor or other health care provider, I can follow through on the plan at home I have a safe and trusting relationship with at least one doctor or other health care provider
SD
Cronbach’s a
Factor loading
.758
.712 .754 .647
% Strongly/somewhat agree
Mean
90.1 89.8
3.11 3.41 3.53
66.1
2.86
75.9
3.07
.96
.736
70.8
3.00
1.14
.679
81.0
3.11 3.09
.68 .89
84.1
3.12
.83
.828
68.4
2.91
1.03
.743
82.9
3.16
.82
.694
82.3
3.23
.88
.771
.95 .76 .75 1.1
.746
.817
.76 .764
Omitted from final scale were four items: “I know what kind of insurance I have” and “I know what health care services I can get using my health insurance” were omitted from Health Access Literacy subscale during factor analysis. “I have a health insurance card that I carry with me” was omitted from health self-efficacy subscale during factor analysis. One item “I know how to make an appointment with a health care provider” was omitted during cognitive testing. SD ¼ standard deviation.
adolescent health that included adolescents. Results indicated that these two subscales are reliable and valid measures of youth health engagement. The five health access literacy items and five self-health efficacy items each demonstrate high internal consistency (Cronbach’s a of .758 and .817, respectively). Further, exploratory factor analysis results showed that in both cases, the five items loaded onto a single factor. The two subscales were positively correlated, indicating that they were both measuring the same underlying concept. Regression analyses showing high association between the engagement subscales and positive experiences of care and receipt of recommended anticipatory guidance indicated that the scales not only were reliable but also had high construct validity. This is the first study to develop and establish reliability and validity for a patient engagement instrument appropriate for adolescents. While a number of existing instruments measure patient and family experiences of health care [2,27,29,30], instruments measuring patient engagement have focused on adults, particularly chronically ill adults [12,15,31,32]. The YEHS! instrument, in contrast, seeks to measure engagement of adolescents, including nonchronically ill adolescents, using items that are developmentally appropriate and that address engagement with aspects of care important to youth. While our conceptualization of youth health engagement is consistent with that of patient engagement among adults, the youth health engagement scale addresses elements of engagement that are particularly pertinent to adolescents. For example, knowledge of confidential services is important for adolescents given their increasing need to access health care on their own. Further, it addresses elements of engagement that will ensure that they develop into adults who can effectively maneuver through the health care system (i.e., knowing where to get care and knowing how to use health insurance).
Adolescent health care and adolescent perspectives on health care have historically received little attention. From 2010 to 2012,
