care, health and development Child: Original Article bs_bs_banner
doi:10.1111/cch.12207
Development and validation of a generic scale for use in transition programmes to measure self-management skills in adolescents with chronic health conditions: the TRANSITION-Q A. F. Klassen,* C. Grant,* R. Barr,† H. Brill,* O. Kraus de Camargo,‡ G. M. Ronen,* M. C. Samaan,* T. Mondal,§ S. J. Cano,¶ A. Schlatman,** E. Tsangaris,** U. Athale,* N. Wickert†† and J. W. Gorter‡ *Department of Pediatrics, McMaster University, Hamilton, ON, Canada †Departments of Pediatrics, Pathology and Medicine, McMaster University, Hamilton, ON, Canada ‡CanChild Centre for Childhood Disability Research and Department of Pediatrics, McMaster University, Hamilton, ON, Canada §Division of Cardiology, Department of Pediatrics, McMaster University, Hamilton, ON, Canada ¶Plymouth University Peninsula Schools of Medicine and Dentistry, Plymouth, Devon, UK **Department of Clinical Epidemiology and Biostatistics, McMaster University, Hamilton, ON, Canada, and ††Department of Social Policy, The London School of Economics and Political Science, London, UK Accepted for publication 18 September 2014
Abstract
Keywords adolescents, psychometrics, Rasch measurement, self-management, transition Correspondence: Anne F. Klassen, DPhil, Department of Pediatrics, McMaster University, 3A, 1280 Main Street West, Hamilton, ON L8S 4K1, Canada E-mail: [email protected] Correction added on 5 December 2014, after first online publication: The author T. Mondal and its corresponding affiliation was initially omitted and has now been added in the author byline.
Aim To develop a generic self-management skills scale for use with adolescents diagnosed with a chronic health condition who are aged 12 to 18 years. Background There is a lack of methodologically sound scales for healthcare teams to use to measure self-management skills in adolescents with chronic conditions transitioning to adult care. Methods Adolescents aged 12 to 18 years with a broad range of chronic health conditions, including neurodevelopmental conditions, were recruited from May to August 2013 from nine outpatient clinics at McMaster Children’s Hospital (Canada). Thirty-two participated in a cognitive interview, and 337 completed a questionnaire booklet. Interviews were used to develop the TRANSITION-Q. Rasch measurement theory (RMT) analysis was used to identify items that represent the best indicators of self-management skills. Traditional psychometric tests of measurement performance were also conducted. Results The response rate was 92% (32/32 cognitive; 337/371 field test). RMT analysis resulted in a 14-item scale with three response options. The overall fit of the observed data to that expected by the Rasch model was non-significant, providing support that this new scale measured a unidimensional construct. Other tests supported the scale as scientifically sound, e.g. Person Separation Index = 0.82; good item fit statistics; no differential item function by age or gender; low residual correlations between items; Cronbach’s alpha = 0.85; test-retest reliability = 0.90; and tests of construct validity that showed, as hypothesized, fewer skills in younger participants and in participants who required assistance to complete the scale. Finally, participants who agreed they are ready to transfer to adult healthcare reported higher TRANSITION-Q scores than did participants who disagreed. Conclusions The TRANSITION-Q is a short, clinically meaningful and psychometrically sound scale. This generic scale can be used in research and in paediatric and adolescent clinics to help evaluate readiness for transition.
© 2014 The Authors. Child: Care, Health and Development published by John Wiley & Sons Ltd. 1 This is an open access article under the terms of the Creative Commons Attribution-NonCommercial-NoDerivs License, which permits use and distribution in any medium, provided the original work is properly cited, the use is non-commercial and no modifications or adaptations are made.
2
A.F. Klassen et al.
Introduction Healthcare transition has been described as the purposeful, planned journey of adolescents with chronic physical and medical conditions from child-centred to adult-oriented healthcare systems (Blum 2002). The American Academy of Pediatrics recognizes that preparing for transition is a process completed over years, and recommends that healthcare providers should use individualized transition plans and start early (preferably 12 years of age) to help adolescents develop selfmanagement skills (American Academy of Pediatrics et al. 2002, 2011). In addition, assessment of transition readiness should be an integral part of any clinical follow-up encounter with adolescents who present with a chronic health condition (Rosen et al. 2003; Grant & Pan 2011). Many paediatric centres, recognizing the importance of preparing adolescents for the inevitable transfer, have developed programmes and clinics specific for this purpose. In Canada, to prepare adolescents for transition, some institutions use transition readiness checklists with patients and their parents. One example is the Good-to-Go transition programme (The Hospital for Sick Children 2011) developed on the basis of the widely adopted ON TRAC transition programme (Paone et al. 2006; Canadian Paediatric Society et al. 2007). An expert panel on transition to adult healthcare of youth with chronic and/or complex health conditions in the province of Ontario called recently for the development of a transition readiness/risk assessment tool for incorporation into paediatric and adolescent medicine clinics to evaluate readiness for transition (Provincial Council for Maternal and Child Health 2013). Such a scale needs to be both clinically meaningful (content covering skills that adolescents need to acquire to be able to care for their health and healthcare as adults) and scientifically sound (valid, reliable and responsive to change). The tool should be composed of a conformable set of items that together map out a clinical hierarchy such that it can be used to track the mastery of skills over time. The use of such a tool by paediatric healthcare providers would make it possible for them to identify the presence or absence of skills based on the pattern of responses to items on the scale, set individualized goals with the youth, apply targeted interventions and reassess to determine if skills have been acquired. Three recent systematic reviews that identified generic and disease-specific transition readiness tools concluded that there is a lack of psychometric evidence supporting the available tools and that further testing and new measures are needed (Stinson et al. 2013; Schwartz et al. 2014; Zhang et al. 2014). Of available generic tools, the Transition Readiness Assessment
Questionnaire (TRAQ; Sawicki et al. 2011) was identified as the most scientifically sound measure. This 29-item scale measures self-management and self-advocacy related to healthcare and other life areas, including education, work and daily life. Some of the content is specific to the United States (e.g. do you apply for health insurance if you lose your current coverage?), making it less relevant in countries, like Canada, with a universal healthcare system. Furthermore, the TRAQ is targeted to older youth (16 to 26 years) and thus not appropriate to use to track the development of transition skills of young adolescents. Following internationally accepted guidelines for the development of a new patient-reported outcome instrument (Aaronson et al. 2002; US Department of Health and Human Services 2009; Patrick et al. 2011a,b), our team previously used modern psychometric methods [i.e. Rasch measurement theory (RMT) (Rasch 1960)] to develop a self-management skills in health and healthcare scale in a sample of childhood cancer survivors (Klassen et al. 2014). The scale’s content was developed from qualitative interviews with 38 survivors and field tested in a sample of 185 survivors aged 15 to 19 years. The scale’s content was designed to measure a single construct (i.e. skills required to manage one’s health and healthcare) rather than a multidimensional construct such as those measured by the TRAQ [i.e. skills required in multiple areas including education, work and daily life (Sawicki et al. 2011)]. The objective of the present study was to adapt the scale developed with childhood cancer survivors for use with younger adolescents diagnosed with a broad range of chronic health conditions. The specific aims of our study were as follows: (1) conduct individual cognitive interviews with adolescents aged 12 to 18 years with different health conditions to obtain their feedback on the scale’s instructions, response options and items, to identify any missing content, and to revise the scale as necessary for use as a generic instrument; (2) conduct a large-scale field test to collect data from a large sample of adolescents, use the data to choose a subset of items that represent the best indicators of self-management skills in health and healthcare, and examine the scale’s scientific properties (reliability and validity).
Methods Local research ethics board approval was obtained prior to starting this study. Subjects included adolescents 12–18 years of age with a diagnosed chronic health condition. Recruitment took place between February and August 2013. During that time, a member of the research team approached eligible participants in the following outpatient clinics at McMaster
© 2014 The Authors. Child: Care, Health and Development published by John Wiley & Sons Ltd, Child: care, health and development
The TRANSITION-Q 3
Children’s Hospital (Canada): adolescent medicine, cardiology, developmental paediatrics, diabetes, gastroenterology, haematology, oncology, neurology and respiratory medicine. The parent signed a consent form for patients under 16 years of age, and adolescents aged 16 and older signed a consent form on their own behalf.
Cognitive interviews Cognitive interviews were used to obtain feedback on the scale and its content. Participants (see Table 1) had to be able to read and understand English to participate. Qualitative methods such as cognitive interviews are valuable methods for tailoring item wording, item format and presentation so they are optimally understood by respondents (Bevans et al. 2010). We tested 19 items in round 1 with 23 participants (see Table 2, column 1). During these face-to-face interviews, participants were asked to read the instructions, which are as follows: ‘These
questions are about being in charge of your health. For each question, please circle only 1 answer’. Following the instructions, participants were asked to read each item in turn, and then to rephrase them in their own words and to classify them as ‘easy’, ‘medium’ or ‘hard’ to understand. For any items judged ‘medium’ or ‘hard’, the participant was asked to suggest ways to reword the item to improve comprehension. For each item, participants were asked to answer using the response options provided (i.e. strongly agree, agree, disagree, strongly disagree), to explain their thought process behind their answer and to comment on whether the response options were appropriate for the item. A second round of interviews was conducted with new nine participants after revising the scale based on round 1 feedback. We tested nine items (see Table 2, column 7). During round 2, participants were shown the original and revised items and response options and asked which they preferred. Feedback obtained from the second round of interviews was used to make final changes to the scale prior to field testing.
Table 1. Characteristics of patients who participated in cognitive interviews and field test
Cognitive interviews (n = 32) Round 1
Gender Male Female Missing Age in years 12 13 14 15 16 17 18 Missing Ethnicity Caucasian Other Missing Health condition Asthma Blood disorder Cancer Cerebral palsy Cystic fibrosis Diabetes Eating disorder Epilepsy Cardiac disorder Inflammatory bowel disease Mental health disorder Other
Field test (n = 337)
Round 2
n
%
n
%
n
%
11 12 0
47.8 52.2 0
5 4 0
55.6 44.4 0
134 202 1
39.8 59.9 0.3
2 4 4 3 3 4 2 1
8.7 17.4 17.4 13.0 13.0 17.4 8.7 4.4
1 1 2 2 1 2 0 0
11.1 11.1 22.2 22.2 11.1 22.2 0 0
26 44 46 67 81 59 13 1
7.7 13.1 13.6 19.9 24.0 17.5 3.9 0.3
15 7 1
65.2 30.4 4.4
8 1 0
88.9 11.1 0
267 58 12
79.2 17.2 3.6
1 0 6 3 0 1 1 0 4 5 0 2
4.4 0 26 13 0 4.4 4.4 0 17.3 21.7 0 8.8
0 0 4 0 0 0 2 0 0 3 0 0
0 0 44.5 0 0 0 22.2 0 0 33.3 0 0
6 13 56 10 7 31 78 10 22 71 23 10
1.8 3.9 16.6 3.0 2.1 9.2 23.1 3.0 6.5 21.1 6.8 3.0
© 2014 The Authors. Child: Care, Health and Development published by John Wiley & Sons Ltd, Child: care, health and development
© 2014 The Authors. Child: Care, Health and Development published by John Wiley & Sons Ltd, Child: care, health and development
20 22 22 20
2.8 3.8 4.1 7.9
3
0
1
3
5
8 1 8
3
3
5
3
2
5
0 1 4
4 3
N/A
0
0
0
0
0 0 3
0
1
0
0
0
0
0 0 0
0 1
Revise
Remove
Retain
Retain
Revise
Remove Retain Remove
Revise
Revise
Revise
Revise
Retain
Revise
Retain Retain Retain
Retain Revise
Decision
I make sure I get health care when I am sick (e.g. go to family doctor or emergency at a hospital). I look for an answer when I have a question about my health.
I drop off or pick up my prescriptions when I need medicine.
I see the doctor or nurse on my own during an appointment.
I summarize my medical history when I am asked to.
I speak to the doctor instead of my parent(s) speaking for me.
I contact a doctor when I need to.
I talk about my health condition to people when I need to.
I help to make decisions about my health.
Round 2 items tested
Note: Four items in round 1 had missing data for the item difficulty question. †Items added to the 15-item cancer survivor scale from the discard pile in the original study. ‡Revised item: ‘I make sure I get medical care when I am sick (e.g. go to family doctor or emergency department at a hospital)’; FK = 10. FK, Flesch Kincaid grade reading level.
17
4.9
20
15 22 11
4.9
I prefer to see a doctor or nurse without my parent(s) with me.
19
18
4.8 4.4 10.1
7.5
I can briefly describe my medical history when asked.
I know how to access medical care when I travel. I book my own doctor’s appointments. I know the type of medical insurance I have (Note: medical insurance pays for things not paid for by the healthcare system). I fill my own prescriptions when I need medicine. I depend on my parent(s) to help me with my health care.† I need my parent(s) to explain what the doctor or nurse says.† My parent(s) sits in the waiting room when I see a doctor or nurse.† I know how to get medical care when I am sick (e.g. go to family doctor or walk-in clinic).†
4.9
20
2.6
I prefer it when a doctor speaks to me instead of my parent(s).
21
4.7
18
6.7
I am in charge of taking any medicine that I need. I know how to contact a doctor if I need to.
22 22 19
3.6 2.3 2.8
I make sure I go to my doctor’s appointments. I ask the doctor or nurse questions. I talk to a doctor or nurse when I have health concerns. I talk about my medical condition to people when I need to.
19 19
5.6 9.6
I answer a doctor’s or nurse’s questions. I participate in making decisions about my health.
Hard
9
9
4.0
7
9
9
8
9
9
9
Easy
8.8
4.8
4.8
7.1
4.8
2.2
4.8
3.6
3.7
FK
0
0
2
0
0
1
0
0
0
Med.
0
0
0
0
0
0
0
0
0
Hard
Original: 1 Revised: 6 Either: 2
Original: 1 Revised: 8
Original: 4 Revised: 5 Original: 2 Revised: 6 Either: 1 Original: 1 Revised: 5 Either: 3 Original: 1 Revised: 7 Either: 1
Original: 2 Revised: 5 Either: 2
Original: 0 Revised: 8 Either: 1
Preferred
Retain
Revise‡
Retain
Retain
Retain
Retain
Retain
Retain
Retain
Decision
Round 1 items tested
Med.
Table 2. Cognitive interview results for each item tested in round 1 and round 2 Easy
A.F. Klassen et al.
FK
4
The TRANSITION-Q 5
Field test Participants who could respond on their own, as well as those who required assistance (i.e. someone to read the items and record the participants’ responses), were invited to participate. Participants were given a questionnaire booklet to complete during their hospital visit. The booklet included the 18-item self-management skills scale alongside demographic (e.g. age, gender) and clinical (e.g. health condition) questions. Data were entered into REDCap, a secure web application for online surveys (www.project-redcap.org). All entered data were checked for accuracy. Participants were asked to participate in a test-retest (TRT) reliability study. For those who agreed, an email with a link to complete the 18-item scale online was sent 2 weeks after the initial assessment with a reminder sent 2 days later.
Item fit can be examined using a number of statistical and graphical tests within Rumm2030 software. Specifically, we examined the following 3 fit indicators: (1) fit residuals (item– person interaction); (2) chi-squared values (item–trait interaction); (3) item characteristic curves (ICC). As a guide, fit residual should fall between −2.5 and +2.5, and chi-squared values should be non-significant after Bonferroni adjustment (Wright & Stone 1979).
Targeting The items that make up a scale should be targeted to the patient population for which the scale was designed. Targeting is determined by the match between the range of a construct measured by the items in a scale, and the range of the construct reported by a sample. The relationship between these two distributions can show how well the scale is targeted to a sample.
Psychometric analysis Validity and reliability was examined using RMT methods (Wright & Masters 1982; Andrich 2004) within RUMM2030 software (Andrich & Sheridan 1997–2011). RMT analysis is a modern psychometric method used increasingly to develop rating scales (Hobart & Cano 2009). This method examines the difference between observed and predicted item responses to determine whether the data collected from a sample ‘fit’ a mathematical model. To determine model fit, a range of statistical and graphical tests are examined, with the evidence from these tests considered together to make decisions about the overall quality of a scale (Rasch 1960; Andrich 1988; Hobart & Cano 2009). The following RMT tests of validity and reliability were performed.
Stability Tests of differential item functioning (DIF) can be used to look at each item in a scale to determine whether the item’s performance is stable across subgroups within a sample. Statistically significant chi-squared values after Bonferroni adjustment indicate potential DIF (Hagquist & Andrich 2004). We examined DIF for gender and age.
Person Separation Index (PSI) Reliability was assessed using the PSI (Andrich 1982), which is equivalent to the Cronbach’s alpha statistic (Cronbach 1951). Values > 0.70 indicate adequate reliability (Nunnally & Berstein 1994; Terwee et al. 2007; Stinson et al. 2013).
Thresholds for item response options A threshold, the location between two adjacent response options, is the point at which the probability of responding is 50% (Andrich 1988). When the response options work in the manner intended, the thresholds should be ordered, which means that respondents can discriminate between response options for each item. We tested this assumption by examining the ordering of thresholds.
Dependency The responses to any item in a scale should not directly influence the response to any other item. As a benchmark, residual correlations below 0.30 are preferred (Marais & Andrich 2008).
Overall model fit Item fit statistics The items of a scale must work together (fit) and map out a clinically important hierarchy for the construct being measured.
An overall chi-squared test of model fit was computed. A nonsignificant P-value indicates that the data provided by participants satisfies the requirements of the Rasch model (Hobart & Cano 2009).
© 2014 The Authors. Child: Care, Health and Development published by John Wiley & Sons Ltd, Child: care, health and development
6
A.F. Klassen et al.
In addition to RMT analyses, we examined scale reliability and validity using the following Classical test theory (CTT) tests: data quality (percent missing data for each item), scaling assumptions (similarity of item means and variances, and magnitude and similarity of corrected item-total correlations; Likert 1932; Ware et al. 1996; McHorney et al. 1997), scale-to-sample targeting [score means, standard deviation (SD), floor and ceiling effects], internal consistency reliability (Cronbach’s alpha; Cronbach 1951) and stability (TRT reliability; Giraudeau & Mary 2001). A minimum standard for Cronbach’s alpha coefficients and TRT reliability is 0.70 (Nunnally & Berstein 1994; Terwee et al. 2007; Stinson et al. 2013). For construct validity, we tested the following three hypotheses: (1) that older participants would report higher scores than younger participants; (2) that participants who completed the scale on their own would report higher scores than participants who required assistance; (3) that scores will be incrementally higher based on participants answer (i.e. ‘strongly disagree’, ‘disagree’, ‘agree’, ‘strongly agree’) to the following statement: ‘I am ready to transfer to adult health care’. For these three tests, we used the Rasch-based person measure scores transformed on a scale from 0 to 100, with higher scores indicative of more engagement in selfmanagement skills.
nine participants. Most participants preferred the new wording of the revised items over the original wording. The new response option format was preferred by eight of the nine participants, with the remaining participant liking both sets of response options equally. Following round 2, one item was revised and a new item was added, i.e. ‘I travel on my own to a doctor’s appointment’, forming an 18-item scale for the field test.
Field test A total of 371 eligible patients were invited to participate and 337 agreed. Non-respondents were more likely to be male (P = 0.04 on Fisher’s exact test), but did not differ by age. Table 2 shows the characteristics of the 337 participants. Most participants (n = 310; 92%) completed the questionnaire booklet unassisted. The 27 participants who required assistance tended to have a health condition that involved some degree of cognitive impairment.
RMT analysis results
Results Cognitive interviews Table 1 shows sample characteristics.
Round 1 For the first round of cognitive interview (see Table 2 columns 1 to 6), the Flesch–Kincaid grade level (Flesch 1948), which measures reading comprehension, ranged from 2.3 to 10.1, with 14 items below grade 6. All 23 participants found the instructions easy to understand, thus they were left unchanged. Fourteen participants indicated that they did not like the agree/ disagree response option format. Based on this feedback, eight items were kept without revision, eight items were revised, three items were dropped, one new item was added and the response option format was changed from agreement to frequency, i.e. ‘never’, ‘sometimes’, ‘often’ and ‘always’.
Round 2 For the second round of cognitive interviews (see Table 2, columns 7 to 13), seven items tested were judged easy by all
Response options Item response option thresholds were ordered for 13 of 18 items. Figure 1a shows the category probability curve for one of the problematic items, i.e. ‘I see the doctor or nurse on my own during an appointment’. For this item, respondents were not able to distinguish between the four response options, i.e. at no point on the continuum did response option 2 have the highest probability of being chosen. Instead, participants whose self-management skills location on the scale at this point in the continuum (often) were actually more likely to choose either of the two adjacent categories (sometimes or always). Figure 1b shows this same item after rescoring. To simplify the scoring and ensure ordered thresholds, we re-scored all items into three response options, i.e. never = 0; sometimes or often = 1; always = 2. Subsequent RMT analyses for this scale used the re-scored data. Item fit statistics We dropped four items, due to poor item fit, DIF by age and/or the need to reverse score. The remaining 14 items had item fit that met the criteria, with fit residuals within the recommended range of −2.5 to +2.5, and chi-squared P-values that were not significant after Bonferroni adjustment (see Table 3).
© 2014 The Authors. Child: Care, Health and Development published by John Wiley & Sons Ltd, Child: care, health and development
The TRANSITION-Q 7
Figure 1. (a) Category probability curves for item ‘I see the doctor or nurse on my own during an appointment’ before rescoring. The x-axis represents the construct of self-management skills with higher scores increasing to the right. The y-axis shows the probability of endorsing the response categories, reading left to right: 0 (first curve) ‘never’, 1 (second curve) ‘sometimes’, 2 (third curve) ‘often’ and 3 (fourth curve) ‘always’. (b) The item ‘I see the doctor or nurse on my own during an appointment’ after rescoring. Here, the y-axis shows the probability of endorsing the response categories, reading left to right: 0 (first curve) ‘never’, 1 (second curve) ‘sometimes’ or ‘often’, 2 (third curve) ‘always’.
Table 3. Statistical indicators of fit, including fit residual and chi-square Item description
Item location
SE
Fit residual
d.f.
Chi-square
d.f.
P-value
I answer a doctor’s or nurse’s questions. I help to make decisions about my health. I am in charge of taking any medicine that I need. I talk to a doctor or nurse when I have health concerns. I look for an answer when I have a question about my health. I talk about my health condition to people when I need to. I ask the doctor or nurse questions. I speak to the doctor instead of my parent(s) speaking for me. I summarize my medical history when I am asked to. I contact a doctor when I need to. I see the doctor or nurse on my own during an appointment. I drop off or pick up my prescriptions when I need medicine. I travel on my own to a doctor’s appointment. I book my own doctor’s appointments.
−2.41 −1.21 −0.99 −0.84 −0.76 −0.74 −0.33 −0.28 −0.15 0.39 0.61 1.57 2.52 2.62
0.12 0.11 0.10 0.12 0.11 0.11 0.12 0.11 0.10 0.10 0.09 0.10 0.12 0.13
1.16 0.03 2.12 −0.85 −0.91 1.30 −0.61 −1.04 −1.75 −1.29 1.94 −0.02 −0.33 −1.40
300 310 309 310 311 310 309 309 311 310 309 309 311 310
4.56 4.77 16.13 6.57 5.92 4.92 2.64 8.21 5.74 5.47 2.65 5.32 6.66 8.62
5 5 5 5 5 5 5 5 5 5 5 5 5 5
0.47 0.44 0.01 0.26 0.31 0.43 0.76 0.15 0.33 0.36 0.75 0.39 0.25 0.13
d.f., degrees of freedom; SE, standard error.
Targeting Figure 2 shows the distribution of person measurements (top histogram) and item locations (bottom histogram) and provides evidence that our study sample lies inside the range in which the scale provides measurement. The scale thus defined a continuum for self-management skills and was well targeted to the sample of participants.
Stability DIF was not detected for gender or age after Bonferroni correction (results not shown but available by contacting the corresponding author).
PSI The PSI value was 0.82 indicating good reliability.
© 2014 The Authors. Child: Care, Health and Development published by John Wiley & Sons Ltd, Child: care, health and development
8
A.F. Klassen et al.
Figure 2. Person-item thresholds distribution. The x-axis represents the construct (self-management skills), with higher scores (more skills) increasing to the right. The y-axis shows the frequency of person measure locations (top histogram) and item locations (bottom histogram).
Dependency Residual correlations for 12/14 items were below 0.30, supporting local independence. Residual correlations for the two items that were marginally above 0.30 were sub-tested and found to have little impact on reliability, i.e. PSI was 0.80 after sub-testing. Overall model fit A non-significant chi-squared value was achieved (chi squared = 88.17, degrees of freedom = 70, P-value = 0.07), supporting data fit to the Rasch model.
CTT analysis results Flesch–Kincaid grade level for the 14 items ranged from 2.2 to 7.1 (average 4.4), with one item above grade 6. Data quality was high, with little item level missing data (range 0 to 3.6%). Raw scale total scores were computable for the 90.2% (i.e. participants who provided answers to all 14 items). Scaling assumptions and scale-to-sample targeting (see Table 4) were appropriate given the response pattern for a scale designed to measure a clinical hierarchy, e.g. the pattern of responses for floor/ceiling effect ranged from 1/48 (easiest answer to endorse) to 78/3 (hardest item to endorse). More specifically, the easiest skill for participants to endorse was ‘I answer a doctor’s or nurse’s question’. This item has the highest mean score (2.31) and largest ceiling effect. The most difficult item to endorse was ‘I book my own doctor’s appointments. This skill had the lowest mean score (0.35) with the smallest ceiling effect (3%).
Internal consistency reliability was high, with a Cronbach’s alpha of 0.85. For the TRT, 220 (65.3%) participants agreed to participate but only 47 completed a second copy of the scale online (response rate of 21.4%). The number of days between the test and re-test ranged from 14 to 54; 87.2%of re-tests were completed by day 21. The intra-class correlation coefficient for the 47 subjects was 0.90 and for those who responded by day 21 was 0.92. The mean scale score was lower for participants who were younger (Pearson correlation = 0.46; P < 0.01), and who needed assistance to complete the scale (mean = 52.7, SD = 12.4 versus mean = 43.3, SD = 13.3; P < 0.01 on t-test). Finally, the mean score was incrementally higher (P < 0.01 on analysis of variance) according to the strength of agreement to the item: ‘I am ready to transfer to adult health care’ (see Fig. 3).
Discussion It is recommended that adolescents with chronic health conditions should prepare for transition in early adolescence (American Academy of Pediatrics et al. 2002, 2011; Gorter et al. 2011; McManus et al. 2013). However, given the lack of validated transition tools available to date, it is not surprising that many transition programmes in Canada have incorporated ad hoc checklists into clinical practice. Ad hoc measures may pose reasonable questions and serve as a practical clinical guide, but we cannot be confident about the reliability (i.e. ability to
© 2014 The Authors. Child: Care, Health and Development published by John Wiley & Sons Ltd, Child: care, health and development
The TRANSITION-Q 9
Table 4. Traditional psychometric methods including data quality, scaling assumptions, targeting and reliability
Item description I answer a doctor’s or nurse’s questions. I help to make decisions about my health. I am in charge of taking any medicine that I need. I talk to a doctor or nurse when I have health concerns. I look for an answer when I have a question about my health. I talk about my health condition to people when I need to. I ask the doctor or nurse questions. I speak to the doctor instead of my parent(s) speaking for me. I summarize my medical history when I am asked to. I contact a doctor when I need to. I see the doctor or nurse on my own during an appointment. I drop off or pick up my prescriptions when I need medicine. I travel on my own to a doctor’s appointment. I book my own doctor’s appointments.
Data quality
Scaling assumptions
Missing data (%)
Possible score range
Actual score range
Mean score
SD
CITC
3.6 0.3 0.3 0.3
0–3 0–3 0–3 0–3
0–3 0–3 0–3 0–3
2.31 2.07 2.17 1.88
0.76 0.86 0.98 0.90
0.42 0.52 0.32 0.57
1/48 5/36 8/50 6/29
−0.67 −0.55 −0.87 −0.27
0
0–3
0–3
1.95
0.93
0.55
8/33
−0.50
0.3
0–3
0–3
1.86
0.92
0.42
7/29
−0.23
0.6 0.6
0–3 0–3
0–3 0–3
1.60 1.67
0.89 0.93
0.53 0.65
8/19 10/22
0.17 −0.08
0 0.3 0.6
0–3 0–3 0–3
0–3 0–3 0–3
1.69 1.40 1.22
1.06 1.10 1.09
0.62 0.57 0.43
17/28 27/21 33/18
−0.23 0.13 0.40
0.6
0–3
0–3
0.63
0.99
0.47
64/10
1.42
0.3 0.3
0–3 0–3
0–3 0–3
0.29 0.35
0.70 0.73
0.41 0.47
82/3 78/3
2.59 2.16
100
80
60
40
20
0 Definitely Disagree Somewhat Disagree
Somewhat Agree
Definitely Agree
Figure 3. Mean TRANSITION-Q score for participants by strength of agreement to the statement: I am ready to transfer to adult health care.
produce consistent and reproducible scores) or validity (i.e. ability to measure what is intended to be measured) of any measure that has not been formally developed or tested. The TRANSITION-Q represents a new psychometrically sound and clinically meaningful scale that can be used in transition programmes with adolescents starting at 12 years of age
Targeting Floor/ceiling (%)
Skewness
to measure and track the development of skills they need to acquire to manage their health and healthcare. The content of the scale was designed to include a range of skills that vary from those that even young adolescents should be able to do (e.g. answer a doctor’s or nurse’s questions) to skills that may require instruction or training (e.g. book a clinical appointment). The use of this scale in clinical practice by healthcare providers (e.g., with electronic data collection and real-time generation of patient reports showing the pattern of responses to the 14 items) would make it possible to identify patients’ strengths as well as areas for improvement so that these can be addressed (Greenhalgh & Meadows 1999; Marshall et al. 2006; Valderas et al. 2008). The scale can be used also in transition readiness research where studies are needed to document the outcomes of selfmanagement interventions; a recent systematic review of selfmanagement interventions for children and youth with disabilities found that only six studies of distinct programmes were published in a 30-year period providing only limited evidence that self-management interventions are effective (Lindsay et al. 2014). Self-management skills represent just one of an array of factors hypothesized to influence transition readiness in the Social Ecological Model of Adolescents and Young Adults Readiness for Transition (Schwartz et al. 2011). This model accounts for a range of objective modifiable subjective factors important in the process of transition for patients with
© 2014 The Authors. Child: Care, Health and Development published by John Wiley & Sons Ltd, Child: care, health and development
10
A.F. Klassen et al.
chronic illnesses (e.g. knowledge, skills, self-efficacy, beliefs/ expectations, goals, relationships, psychosocial functioning). This framework provides an evidence-based framework that could be used to guide the development and implementation of transition readiness research. It would be interesting to investigate the relationship between the specific skills measured by the TRANSITION-Q and skills in various life domains, as in the end, all these crucial life skills are required for adolescents to achieve their life goals and experience a successful transition. Our study has several limitations. While an important strength is the large sample size recruited from nine different subspecialty clinics, some health conditions in our sample were under-represented (e.g. asthma, cystic fibrosis) and some common health conditions were not included (e.g. arthritis). Research to examine the psychometric properties of the TRANSITION-Q in under-represented and additional patient groups will be required. Second, the study sample was recruited from a single institution, and we recognize that the inclusion of more hospitals may have increased the heterogeneity of the sample even further. Third, while the response rate was high and shows that adolescents are willing to complete a short questionnaire during the course of a hospital visit, the response to the additional TRT survey was low. It is possible adolescents approached in our study may not have been comfortable saying no when asked face-to-face to complete the TRT at home, or that an email with a link to an online survey is not an optimal method for data collection from adolescents. Finally, our scale measures a single construct, i.e. skills required to manage one’s health and healthcare. We recognize that the development of self-management skills outside of healthcare (e.g. education, work, daily life) are also important to transition success, as are other modifiable subjective variables (Schwartz et al. 2011) such as the concept of self-efficacy (i.e. a person’s belief in his or her ability to succeed in specific situations (Bandura 1977). As such, there is scope for future research to develop additional clinically meaningful scales to measure other important transition-related constructs. A new study to further test the TRANSITION-Q across six paediatric healthcare centres in Ontario for further implementation and validation is now in preparation. In conclusion, we have provided evidence for reliability and validity of the TRANSITION-Q. While there is a need for further psychometric work to be carried out to add to the evidence base for the use of this scale and the generalizability of its measurement properties, we argue that the clinical meaning of the scale’s scores can be best established when the TRANSITION-Q tool is used and evaluated broadly.
Key messages • Three recent systematic reviews identified a lack of methodologically sound transition readiness scales. • The TRANSITION-Q is a reliable and valid 14-item scale that measures self-management skills in health and healthcare in adolescents aged 12 to 18 years across a broad range of chronic health conditions. • The TRANSITION-Q was designed using a modern psychometric approach to rating scale development and validation, i.e. Rasch measurement theory analysis. • The TRANSITION-Q can be used in clinical practice by healthcare professionals to help adolescents develop the skills needed to take care of their health and healthcare needs as adults. • The TRANSITION-Q can be used in research studies to understand factors that influence transition readiness.
Funding Anne Klassen holds a Canadian Institutes of Health Research Mid-Career Award, which provides a research stipend that was used alongside matched funding from the Department of Pediatrics at McMaster University to fund this study. Jan Willem Gorter holds the Scotiabank Chair in Child Health Research (2013–2017).
Conflict of interests None of the authors has any conflicts of interest to declare.
References Aaronson, N., Alonso, J., Burnam, A., Lohr, K. N., Patrick, D. L., Perrin, E. & Stein, R. E. (2002) Assessing health status and quality-of-life instruments: attributes and review criteria. Quality of Life Research: An International Journal of Quality of Life Aspects of Treatment, Care and Rehabilitation, 11, 193–205. American Academy of Pediatrics, American Academy of Family Physicians & American College of Physicians-American Society of Internal Medicine (2002) A consensus statement on health care transitions for young adults with special health care needs. Pediatrics, 110, 1304–1306. American Academy of Pediatrics, American Academy of Family Physicians, American College of Physicians, Transitions Clinical Report Authoring Group, Cooley, W. C. & Sagerman, P. J. (2011) Supporting the health care transition from adolescence to adulthood in the medical home. Pediatrics, 128, 182–200.
© 2014 The Authors. Child: Care, Health and Development published by John Wiley & Sons Ltd, Child: care, health and development
The TRANSITION-Q 11
Andrich, D. (1982) An index of person separation in latent trait theory, the traditional KR20 index, and the Guttman scale response pattern. Education Research Perspectives, 9, 95–104. Andrich, D. (1988) Rasch Models for Measurement. Sage, Newbury Park, CA, USA. Andrich, D. (2004) Controversy and the Rasch model: a characteristic of incompatible paradigms? Medical Care, 42, I7–16. Andrich, D. & Sheridan, B. (1997–2011) RUMM2030. RUMM Laboratory, Perth, Australia. Bandura, A. (1977) Self-efficacy: towards a unifying theory of behavioral change. Psychological Review, 84, 191–215. Bevans, K. B., Riley, A. W., Moon, J. & Forrest, C. B. (2010) Conceptual and methodological advances in child-reported outcomes measurement. Expert Review of Pharmacoeconomics & Outcomes Research, 10, 385–386. Blum, R. W. (2002) Improving transition for adolescents with special health care needs from pediatric to adult-centered health care. Pediatrics, 110, 1301–1303. Canadian Paediatric Society, Adolescent Health Committee, [Principal authors: Kaufman, M. & Pinzon, J.] (2007) Transition to adult care for youth with special health care needs. Paediatrics and Child Health, 12, 785–788. Cronbach, L. (1951) Coefficient alpha and the internal structure of tests. Psychometrika, 16, 297–334. Flesch, R. (1948) A new readability yardstick. Journal of Applied Psychology, 32, 221–233. Giraudeau, B. & Mary, J. Y. (2001) Planning a reproducibility study: how many subjects and how many replicates per subject for an expected width of the 95 per cent confidence interval of the intraclass correlation coefficient. Statistics in Medicine, 20, 3205–3214. Gorter, J. W., Stewart, D. & Woodbury-Smith, M. (2011) Youth in transition: care, health and development. Child: Care, Health and Development, 37, 757–763. Grant, C. & Pan, J. (2011) A comparison of five transition programmes for youth with chronic illness in Canada. Child: Care, Health and Development, 37, 815–820. Greenhalgh, J. & Meadows, K. (1999) The effectiveness of the use of patient-based measures of health in routine practice in improving the process and outcomes of patient care: a literature review. Journal of Evaluation in Clinical Practice, 5, 401–416. Hagquist, C. & Andrich, D. (2004) Is the sense of coherence-instrument applicable on adolescents? A latent trait analysis using Rasch-modelling. Personality and Individual Differences, 36, 955–968. Hobart, J. & Cano, S. (2009) Improving the evaluation of therapeutic interventions in multiple sclerosis: the role of new psychometric methods. Health Technology Assessment, 13, 1–200. Klassen, A. F., Rosenberg-Yunger, Z. R., D’Agostino, N. M., Cano, S. J., Barr, R., Syed, I., Granek, L., Greenberg, M. L., Dix, D. & Nathan, P. C. (2014) The development of scales to measure childhood cancer survivors’ readiness for transition to long-term follow-up care as adults. Health Expectations, July 22 [Epub ahead of print].
Likert, R. (1932) A technique for the measurement of attitudes. Archives of Psychology, 140, 1–55. Lindsay, S., Kingsnorth, S., McDougall, C. & Keating, H. (2014) A systematic review of self-management interventions for children and youth with physical disabilities. Disability and Rehabilitation, 36, 276–288. Marais, I. & Andrich, D. (2008) Formalizing dimension and response violations of local independence in the unidimensional Rasch model. Journal of Applied Measurement, 9, 200–215. Marshall, S., Haywood, K. & Fitzpatrick, R. (2006) Impact of patient-reported outcome measures on routine practice: a structured review. Journal of Evaluation in Clinical Practice, 12, 559–568. McHorney, C. A., Haley, S. M. & Ware, J. E., Jr (1997) Evaluation of the MOS SF-36 Physical Functioning Scale (PF-10): II. Comparison of relative precision using Likert and Rasch scoring methods. Journal of Clinical Epidemiology, 50, 451–461. McManus, M. A., Pollack, L. R., Cooley, W. C., McAllister, J. W., Lotstein, D., Strickland, B. & Mann, M. Y. (2013) Current status of transition preparation among youth with special needs in the United States. Pediatrics, 131, 1090–1097. Nunnally, J. C. & Berstein, I. H. (1994) Psychometric Theory, 3rd edn. McGraw-Hill, New York, NY, USA. Paone, M. C., Wigle, M. & Saewyc, E. (2006) The ON TRAC model for transitional care of adolescents. Progress in Transplantation, 16, 291–302. Patrick, D. L., Burke, L. B., Gwaltney, C. J., Leidy, N. K., Martin, M. L., Molsen, E. & Ring, L. (2011a) Content validity – establishing and reporting the evidence in newly developed patient-reported outcomes (PRO) instruments for medical product evaluation: ISPOR PRO good research practices task force report: part 1 – eliciting concepts for a new PRO instrument. Value in Health, 14, 967–977. Patrick, D. L., Burke, L. B., Gwaltney, C. J., Leidy, N. K., Martin, M. L., Molsen, E. & Ring, L. (2011b) Content validity – establishing and reporting the evidence in newly developed patient-reported outcomes (PRO) instruments for medical product evaluation: ISPOR PRO Good Research Practices Task Force report: part 2-assessing respondent understanding. Value in Health, 14, 978–988. Provincial Council for Maternal and Child Health (2013) Transition to adult healthcare services work group report. Available at: http:// www.pcmch.on.ca/initiatives/transition-adult-healthcare-services (last accessed 24 April 2014). Rasch, G. (1960) Probabilistic Models for Some Intelligence and Attainment Tests. Danish Institute for Education Research, Copenhagen, Denmark. Rosen, D. S., Blum, R. W., Britto, M., Sawyer, S. M., Siegel, D. M. & Society for Adolescent Medicine (2003) Transition to adult health care for adolescents and young adults with chronic conditions: position paper of the Society for Adolescent Medicine. Journal of Adolescent Health, 33, 309–311. Sawicki, G. S., Lukens-Bull, K., Yin, X., Demars, N., Huang, I. C., Livingood, W., Reiss, J. & Wood, D. (2011) Measuring the
© 2014 The Authors. Child: Care, Health and Development published by John Wiley & Sons Ltd, Child: care, health and development
12
A.F. Klassen et al.
transition readiness of youth with special healthcare needs: validation of the TRAQ – Transition Readiness Assessment Questionnaire. Journal of Pediatric Psychology, 36, 160–171. Schwartz, L. A., Tuchman, L. K., Hobbie, W. L. & Ginsberg, J. P. (2011) A social-ecological model of readiness for transition to adult-oriented care for adolescents and young adults with chronic health conditions. Child: Care, Health and Development, 37, 883–895. Schwartz, L. A., Daniel, L. C., Brumley, L. D., Barakat, L. P., Wesley, K. M. & Tuchman, L. K. (2014) Measures of readiness to transition to adult health care for youth with chronic physical health conditions: a systematic review and recommendations for measurement testing and development. Journal of Pediatric Psychology, 39, 588–601. Stinson, J., Kohut, S. A., Spiegel, L., White, M., Gill, N., Colbourne, G., Sigurdson, S., Duffy, K. W., Tucker, L., Stringer, E., Hazel, B., Hochman, J., Reiss, J. & Kaufman, M. (2013) A systematic review of transition readiness and transfer satisfaction measures for adolescents with chronic illness. International Journal of Adolescent Medicine and Health, 6, 1–16. [Epub ahead of print]. Terwee, C. B., Bot, S. D., de Boer, M. R., van der Windt, D. A., Knol, D. L., Dekker, J., Bouter, L. M. & de Vet, H. C. (2007) Quality criteria were proposed for measurement properties of health status questionnaires. Journal of Clinical Epidemiology, 60, 34–42. The Hospital for Sick Children (2011) Readiness checklist. Available at: http://www.sickkids.ca/Good2Go/Transition-Interventions
-Tools/Readiness-checklists/index.html (last accessed 9 December 2012). US Department of Health and Human Services (2009) Guidance for industry: patient reported outcome measures: use in medical product development to support labeling claims. Available at: http:// www.ispor.org/workpaper/FDA%20PRO%20Guidance.pdf (last accessed 24 April 2014). Valderas, J. M., Kotzeva, A., Espallargues, M., Guyatt, G., Ferrans, C. E., Halyard, M. Y., Revicki, D. A., Symonds, T., Parada, A. & Alonso, J. (2008) The impact of measuring patient-reported outcomes in clinical practice: a systematic review of the literature. Quality of Life Research: An International Journal of Quality of Life Aspects of Treatment, Care and Rehabilitation, 17, 179–193. Ware, J. E., Jr, Harris, W. J., Gandek, B., Rogers, B. W. & Reese, P. R. (1996) MAP-R for Windows: Multitrait/Multi-Item Analysis Program – Revised User’s Guide. Health Assessment Laboratory, Boston, MA, USA. Wright, B. & Masters, G. (1982) Rating Scale Analysis: Rasch Measurement. MESA, Chicago, IL, USA. Wright, B. D. & Stone, M. H. (1979) Best Test Design: Rasch Measurement, pp. 1–17. MESA, Chicago, IL, USA. Zhang, L. F., Ho, J. S. & Kennedy, S. E. (2014) A systematic review of the psychometric properties of transition readiness assessment tools in adolescents with chronic disease. BMC Pediatrics, 14, 4.
© 2014 The Authors. Child: Care, Health and Development published by John Wiley & Sons Ltd, Child: care, health and development
doi:10.1111/cch.12207
Development and validation of a generic scale for use in transition programmes to measure self-management skills in adolescents with chronic health conditions: the TRANSITION-Q A. F. Klassen,* C. Grant,* R. Barr,† H. Brill,* O. Kraus de Camargo,‡ G. M. Ronen,* M. C. Samaan,* T. Mondal,§ S. J. Cano,¶ A. Schlatman,** E. Tsangaris,** U. Athale,* N. Wickert†† and J. W. Gorter‡ *Department of Pediatrics, McMaster University, Hamilton, ON, Canada †Departments of Pediatrics, Pathology and Medicine, McMaster University, Hamilton, ON, Canada ‡CanChild Centre for Childhood Disability Research and Department of Pediatrics, McMaster University, Hamilton, ON, Canada §Division of Cardiology, Department of Pediatrics, McMaster University, Hamilton, ON, Canada ¶Plymouth University Peninsula Schools of Medicine and Dentistry, Plymouth, Devon, UK **Department of Clinical Epidemiology and Biostatistics, McMaster University, Hamilton, ON, Canada, and ††Department of Social Policy, The London School of Economics and Political Science, London, UK Accepted for publication 18 September 2014
Abstract
Keywords adolescents, psychometrics, Rasch measurement, self-management, transition Correspondence: Anne F. Klassen, DPhil, Department of Pediatrics, McMaster University, 3A, 1280 Main Street West, Hamilton, ON L8S 4K1, Canada E-mail: [email protected] Correction added on 5 December 2014, after first online publication: The author T. Mondal and its corresponding affiliation was initially omitted and has now been added in the author byline.
Aim To develop a generic self-management skills scale for use with adolescents diagnosed with a chronic health condition who are aged 12 to 18 years. Background There is a lack of methodologically sound scales for healthcare teams to use to measure self-management skills in adolescents with chronic conditions transitioning to adult care. Methods Adolescents aged 12 to 18 years with a broad range of chronic health conditions, including neurodevelopmental conditions, were recruited from May to August 2013 from nine outpatient clinics at McMaster Children’s Hospital (Canada). Thirty-two participated in a cognitive interview, and 337 completed a questionnaire booklet. Interviews were used to develop the TRANSITION-Q. Rasch measurement theory (RMT) analysis was used to identify items that represent the best indicators of self-management skills. Traditional psychometric tests of measurement performance were also conducted. Results The response rate was 92% (32/32 cognitive; 337/371 field test). RMT analysis resulted in a 14-item scale with three response options. The overall fit of the observed data to that expected by the Rasch model was non-significant, providing support that this new scale measured a unidimensional construct. Other tests supported the scale as scientifically sound, e.g. Person Separation Index = 0.82; good item fit statistics; no differential item function by age or gender; low residual correlations between items; Cronbach’s alpha = 0.85; test-retest reliability = 0.90; and tests of construct validity that showed, as hypothesized, fewer skills in younger participants and in participants who required assistance to complete the scale. Finally, participants who agreed they are ready to transfer to adult healthcare reported higher TRANSITION-Q scores than did participants who disagreed. Conclusions The TRANSITION-Q is a short, clinically meaningful and psychometrically sound scale. This generic scale can be used in research and in paediatric and adolescent clinics to help evaluate readiness for transition.
© 2014 The Authors. Child: Care, Health and Development published by John Wiley & Sons Ltd. 1 This is an open access article under the terms of the Creative Commons Attribution-NonCommercial-NoDerivs License, which permits use and distribution in any medium, provided the original work is properly cited, the use is non-commercial and no modifications or adaptations are made.
2
A.F. Klassen et al.
Introduction Healthcare transition has been described as the purposeful, planned journey of adolescents with chronic physical and medical conditions from child-centred to adult-oriented healthcare systems (Blum 2002). The American Academy of Pediatrics recognizes that preparing for transition is a process completed over years, and recommends that healthcare providers should use individualized transition plans and start early (preferably 12 years of age) to help adolescents develop selfmanagement skills (American Academy of Pediatrics et al. 2002, 2011). In addition, assessment of transition readiness should be an integral part of any clinical follow-up encounter with adolescents who present with a chronic health condition (Rosen et al. 2003; Grant & Pan 2011). Many paediatric centres, recognizing the importance of preparing adolescents for the inevitable transfer, have developed programmes and clinics specific for this purpose. In Canada, to prepare adolescents for transition, some institutions use transition readiness checklists with patients and their parents. One example is the Good-to-Go transition programme (The Hospital for Sick Children 2011) developed on the basis of the widely adopted ON TRAC transition programme (Paone et al. 2006; Canadian Paediatric Society et al. 2007). An expert panel on transition to adult healthcare of youth with chronic and/or complex health conditions in the province of Ontario called recently for the development of a transition readiness/risk assessment tool for incorporation into paediatric and adolescent medicine clinics to evaluate readiness for transition (Provincial Council for Maternal and Child Health 2013). Such a scale needs to be both clinically meaningful (content covering skills that adolescents need to acquire to be able to care for their health and healthcare as adults) and scientifically sound (valid, reliable and responsive to change). The tool should be composed of a conformable set of items that together map out a clinical hierarchy such that it can be used to track the mastery of skills over time. The use of such a tool by paediatric healthcare providers would make it possible for them to identify the presence or absence of skills based on the pattern of responses to items on the scale, set individualized goals with the youth, apply targeted interventions and reassess to determine if skills have been acquired. Three recent systematic reviews that identified generic and disease-specific transition readiness tools concluded that there is a lack of psychometric evidence supporting the available tools and that further testing and new measures are needed (Stinson et al. 2013; Schwartz et al. 2014; Zhang et al. 2014). Of available generic tools, the Transition Readiness Assessment
Questionnaire (TRAQ; Sawicki et al. 2011) was identified as the most scientifically sound measure. This 29-item scale measures self-management and self-advocacy related to healthcare and other life areas, including education, work and daily life. Some of the content is specific to the United States (e.g. do you apply for health insurance if you lose your current coverage?), making it less relevant in countries, like Canada, with a universal healthcare system. Furthermore, the TRAQ is targeted to older youth (16 to 26 years) and thus not appropriate to use to track the development of transition skills of young adolescents. Following internationally accepted guidelines for the development of a new patient-reported outcome instrument (Aaronson et al. 2002; US Department of Health and Human Services 2009; Patrick et al. 2011a,b), our team previously used modern psychometric methods [i.e. Rasch measurement theory (RMT) (Rasch 1960)] to develop a self-management skills in health and healthcare scale in a sample of childhood cancer survivors (Klassen et al. 2014). The scale’s content was developed from qualitative interviews with 38 survivors and field tested in a sample of 185 survivors aged 15 to 19 years. The scale’s content was designed to measure a single construct (i.e. skills required to manage one’s health and healthcare) rather than a multidimensional construct such as those measured by the TRAQ [i.e. skills required in multiple areas including education, work and daily life (Sawicki et al. 2011)]. The objective of the present study was to adapt the scale developed with childhood cancer survivors for use with younger adolescents diagnosed with a broad range of chronic health conditions. The specific aims of our study were as follows: (1) conduct individual cognitive interviews with adolescents aged 12 to 18 years with different health conditions to obtain their feedback on the scale’s instructions, response options and items, to identify any missing content, and to revise the scale as necessary for use as a generic instrument; (2) conduct a large-scale field test to collect data from a large sample of adolescents, use the data to choose a subset of items that represent the best indicators of self-management skills in health and healthcare, and examine the scale’s scientific properties (reliability and validity).
Methods Local research ethics board approval was obtained prior to starting this study. Subjects included adolescents 12–18 years of age with a diagnosed chronic health condition. Recruitment took place between February and August 2013. During that time, a member of the research team approached eligible participants in the following outpatient clinics at McMaster
© 2014 The Authors. Child: Care, Health and Development published by John Wiley & Sons Ltd, Child: care, health and development
The TRANSITION-Q 3
Children’s Hospital (Canada): adolescent medicine, cardiology, developmental paediatrics, diabetes, gastroenterology, haematology, oncology, neurology and respiratory medicine. The parent signed a consent form for patients under 16 years of age, and adolescents aged 16 and older signed a consent form on their own behalf.
Cognitive interviews Cognitive interviews were used to obtain feedback on the scale and its content. Participants (see Table 1) had to be able to read and understand English to participate. Qualitative methods such as cognitive interviews are valuable methods for tailoring item wording, item format and presentation so they are optimally understood by respondents (Bevans et al. 2010). We tested 19 items in round 1 with 23 participants (see Table 2, column 1). During these face-to-face interviews, participants were asked to read the instructions, which are as follows: ‘These
questions are about being in charge of your health. For each question, please circle only 1 answer’. Following the instructions, participants were asked to read each item in turn, and then to rephrase them in their own words and to classify them as ‘easy’, ‘medium’ or ‘hard’ to understand. For any items judged ‘medium’ or ‘hard’, the participant was asked to suggest ways to reword the item to improve comprehension. For each item, participants were asked to answer using the response options provided (i.e. strongly agree, agree, disagree, strongly disagree), to explain their thought process behind their answer and to comment on whether the response options were appropriate for the item. A second round of interviews was conducted with new nine participants after revising the scale based on round 1 feedback. We tested nine items (see Table 2, column 7). During round 2, participants were shown the original and revised items and response options and asked which they preferred. Feedback obtained from the second round of interviews was used to make final changes to the scale prior to field testing.
Table 1. Characteristics of patients who participated in cognitive interviews and field test
Cognitive interviews (n = 32) Round 1
Gender Male Female Missing Age in years 12 13 14 15 16 17 18 Missing Ethnicity Caucasian Other Missing Health condition Asthma Blood disorder Cancer Cerebral palsy Cystic fibrosis Diabetes Eating disorder Epilepsy Cardiac disorder Inflammatory bowel disease Mental health disorder Other
Field test (n = 337)
Round 2
n
%
n
%
n
%
11 12 0
47.8 52.2 0
5 4 0
55.6 44.4 0
134 202 1
39.8 59.9 0.3
2 4 4 3 3 4 2 1
8.7 17.4 17.4 13.0 13.0 17.4 8.7 4.4
1 1 2 2 1 2 0 0
11.1 11.1 22.2 22.2 11.1 22.2 0 0
26 44 46 67 81 59 13 1
7.7 13.1 13.6 19.9 24.0 17.5 3.9 0.3
15 7 1
65.2 30.4 4.4
8 1 0
88.9 11.1 0
267 58 12
79.2 17.2 3.6
1 0 6 3 0 1 1 0 4 5 0 2
4.4 0 26 13 0 4.4 4.4 0 17.3 21.7 0 8.8
0 0 4 0 0 0 2 0 0 3 0 0
0 0 44.5 0 0 0 22.2 0 0 33.3 0 0
6 13 56 10 7 31 78 10 22 71 23 10
1.8 3.9 16.6 3.0 2.1 9.2 23.1 3.0 6.5 21.1 6.8 3.0
© 2014 The Authors. Child: Care, Health and Development published by John Wiley & Sons Ltd, Child: care, health and development
© 2014 The Authors. Child: Care, Health and Development published by John Wiley & Sons Ltd, Child: care, health and development
20 22 22 20
2.8 3.8 4.1 7.9
3
0
1
3
5
8 1 8
3
3
5
3
2
5
0 1 4
4 3
N/A
0
0
0
0
0 0 3
0
1
0
0
0
0
0 0 0
0 1
Revise
Remove
Retain
Retain
Revise
Remove Retain Remove
Revise
Revise
Revise
Revise
Retain
Revise
Retain Retain Retain
Retain Revise
Decision
I make sure I get health care when I am sick (e.g. go to family doctor or emergency at a hospital). I look for an answer when I have a question about my health.
I drop off or pick up my prescriptions when I need medicine.
I see the doctor or nurse on my own during an appointment.
I summarize my medical history when I am asked to.
I speak to the doctor instead of my parent(s) speaking for me.
I contact a doctor when I need to.
I talk about my health condition to people when I need to.
I help to make decisions about my health.
Round 2 items tested
Note: Four items in round 1 had missing data for the item difficulty question. †Items added to the 15-item cancer survivor scale from the discard pile in the original study. ‡Revised item: ‘I make sure I get medical care when I am sick (e.g. go to family doctor or emergency department at a hospital)’; FK = 10. FK, Flesch Kincaid grade reading level.
17
4.9
20
15 22 11
4.9
I prefer to see a doctor or nurse without my parent(s) with me.
19
18
4.8 4.4 10.1
7.5
I can briefly describe my medical history when asked.
I know how to access medical care when I travel. I book my own doctor’s appointments. I know the type of medical insurance I have (Note: medical insurance pays for things not paid for by the healthcare system). I fill my own prescriptions when I need medicine. I depend on my parent(s) to help me with my health care.† I need my parent(s) to explain what the doctor or nurse says.† My parent(s) sits in the waiting room when I see a doctor or nurse.† I know how to get medical care when I am sick (e.g. go to family doctor or walk-in clinic).†
4.9
20
2.6
I prefer it when a doctor speaks to me instead of my parent(s).
21
4.7
18
6.7
I am in charge of taking any medicine that I need. I know how to contact a doctor if I need to.
22 22 19
3.6 2.3 2.8
I make sure I go to my doctor’s appointments. I ask the doctor or nurse questions. I talk to a doctor or nurse when I have health concerns. I talk about my medical condition to people when I need to.
19 19
5.6 9.6
I answer a doctor’s or nurse’s questions. I participate in making decisions about my health.
Hard
9
9
4.0
7
9
9
8
9
9
9
Easy
8.8
4.8
4.8
7.1
4.8
2.2
4.8
3.6
3.7
FK
0
0
2
0
0
1
0
0
0
Med.
0
0
0
0
0
0
0
0
0
Hard
Original: 1 Revised: 6 Either: 2
Original: 1 Revised: 8
Original: 4 Revised: 5 Original: 2 Revised: 6 Either: 1 Original: 1 Revised: 5 Either: 3 Original: 1 Revised: 7 Either: 1
Original: 2 Revised: 5 Either: 2
Original: 0 Revised: 8 Either: 1
Preferred
Retain
Revise‡
Retain
Retain
Retain
Retain
Retain
Retain
Retain
Decision
Round 1 items tested
Med.
Table 2. Cognitive interview results for each item tested in round 1 and round 2 Easy
A.F. Klassen et al.
FK
4
The TRANSITION-Q 5
Field test Participants who could respond on their own, as well as those who required assistance (i.e. someone to read the items and record the participants’ responses), were invited to participate. Participants were given a questionnaire booklet to complete during their hospital visit. The booklet included the 18-item self-management skills scale alongside demographic (e.g. age, gender) and clinical (e.g. health condition) questions. Data were entered into REDCap, a secure web application for online surveys (www.project-redcap.org). All entered data were checked for accuracy. Participants were asked to participate in a test-retest (TRT) reliability study. For those who agreed, an email with a link to complete the 18-item scale online was sent 2 weeks after the initial assessment with a reminder sent 2 days later.
Item fit can be examined using a number of statistical and graphical tests within Rumm2030 software. Specifically, we examined the following 3 fit indicators: (1) fit residuals (item– person interaction); (2) chi-squared values (item–trait interaction); (3) item characteristic curves (ICC). As a guide, fit residual should fall between −2.5 and +2.5, and chi-squared values should be non-significant after Bonferroni adjustment (Wright & Stone 1979).
Targeting The items that make up a scale should be targeted to the patient population for which the scale was designed. Targeting is determined by the match between the range of a construct measured by the items in a scale, and the range of the construct reported by a sample. The relationship between these two distributions can show how well the scale is targeted to a sample.
Psychometric analysis Validity and reliability was examined using RMT methods (Wright & Masters 1982; Andrich 2004) within RUMM2030 software (Andrich & Sheridan 1997–2011). RMT analysis is a modern psychometric method used increasingly to develop rating scales (Hobart & Cano 2009). This method examines the difference between observed and predicted item responses to determine whether the data collected from a sample ‘fit’ a mathematical model. To determine model fit, a range of statistical and graphical tests are examined, with the evidence from these tests considered together to make decisions about the overall quality of a scale (Rasch 1960; Andrich 1988; Hobart & Cano 2009). The following RMT tests of validity and reliability were performed.
Stability Tests of differential item functioning (DIF) can be used to look at each item in a scale to determine whether the item’s performance is stable across subgroups within a sample. Statistically significant chi-squared values after Bonferroni adjustment indicate potential DIF (Hagquist & Andrich 2004). We examined DIF for gender and age.
Person Separation Index (PSI) Reliability was assessed using the PSI (Andrich 1982), which is equivalent to the Cronbach’s alpha statistic (Cronbach 1951). Values > 0.70 indicate adequate reliability (Nunnally & Berstein 1994; Terwee et al. 2007; Stinson et al. 2013).
Thresholds for item response options A threshold, the location between two adjacent response options, is the point at which the probability of responding is 50% (Andrich 1988). When the response options work in the manner intended, the thresholds should be ordered, which means that respondents can discriminate between response options for each item. We tested this assumption by examining the ordering of thresholds.
Dependency The responses to any item in a scale should not directly influence the response to any other item. As a benchmark, residual correlations below 0.30 are preferred (Marais & Andrich 2008).
Overall model fit Item fit statistics The items of a scale must work together (fit) and map out a clinically important hierarchy for the construct being measured.
An overall chi-squared test of model fit was computed. A nonsignificant P-value indicates that the data provided by participants satisfies the requirements of the Rasch model (Hobart & Cano 2009).
© 2014 The Authors. Child: Care, Health and Development published by John Wiley & Sons Ltd, Child: care, health and development
6
A.F. Klassen et al.
In addition to RMT analyses, we examined scale reliability and validity using the following Classical test theory (CTT) tests: data quality (percent missing data for each item), scaling assumptions (similarity of item means and variances, and magnitude and similarity of corrected item-total correlations; Likert 1932; Ware et al. 1996; McHorney et al. 1997), scale-to-sample targeting [score means, standard deviation (SD), floor and ceiling effects], internal consistency reliability (Cronbach’s alpha; Cronbach 1951) and stability (TRT reliability; Giraudeau & Mary 2001). A minimum standard for Cronbach’s alpha coefficients and TRT reliability is 0.70 (Nunnally & Berstein 1994; Terwee et al. 2007; Stinson et al. 2013). For construct validity, we tested the following three hypotheses: (1) that older participants would report higher scores than younger participants; (2) that participants who completed the scale on their own would report higher scores than participants who required assistance; (3) that scores will be incrementally higher based on participants answer (i.e. ‘strongly disagree’, ‘disagree’, ‘agree’, ‘strongly agree’) to the following statement: ‘I am ready to transfer to adult health care’. For these three tests, we used the Rasch-based person measure scores transformed on a scale from 0 to 100, with higher scores indicative of more engagement in selfmanagement skills.
nine participants. Most participants preferred the new wording of the revised items over the original wording. The new response option format was preferred by eight of the nine participants, with the remaining participant liking both sets of response options equally. Following round 2, one item was revised and a new item was added, i.e. ‘I travel on my own to a doctor’s appointment’, forming an 18-item scale for the field test.
Field test A total of 371 eligible patients were invited to participate and 337 agreed. Non-respondents were more likely to be male (P = 0.04 on Fisher’s exact test), but did not differ by age. Table 2 shows the characteristics of the 337 participants. Most participants (n = 310; 92%) completed the questionnaire booklet unassisted. The 27 participants who required assistance tended to have a health condition that involved some degree of cognitive impairment.
RMT analysis results
Results Cognitive interviews Table 1 shows sample characteristics.
Round 1 For the first round of cognitive interview (see Table 2 columns 1 to 6), the Flesch–Kincaid grade level (Flesch 1948), which measures reading comprehension, ranged from 2.3 to 10.1, with 14 items below grade 6. All 23 participants found the instructions easy to understand, thus they were left unchanged. Fourteen participants indicated that they did not like the agree/ disagree response option format. Based on this feedback, eight items were kept without revision, eight items were revised, three items were dropped, one new item was added and the response option format was changed from agreement to frequency, i.e. ‘never’, ‘sometimes’, ‘often’ and ‘always’.
Round 2 For the second round of cognitive interviews (see Table 2, columns 7 to 13), seven items tested were judged easy by all
Response options Item response option thresholds were ordered for 13 of 18 items. Figure 1a shows the category probability curve for one of the problematic items, i.e. ‘I see the doctor or nurse on my own during an appointment’. For this item, respondents were not able to distinguish between the four response options, i.e. at no point on the continuum did response option 2 have the highest probability of being chosen. Instead, participants whose self-management skills location on the scale at this point in the continuum (often) were actually more likely to choose either of the two adjacent categories (sometimes or always). Figure 1b shows this same item after rescoring. To simplify the scoring and ensure ordered thresholds, we re-scored all items into three response options, i.e. never = 0; sometimes or often = 1; always = 2. Subsequent RMT analyses for this scale used the re-scored data. Item fit statistics We dropped four items, due to poor item fit, DIF by age and/or the need to reverse score. The remaining 14 items had item fit that met the criteria, with fit residuals within the recommended range of −2.5 to +2.5, and chi-squared P-values that were not significant after Bonferroni adjustment (see Table 3).
© 2014 The Authors. Child: Care, Health and Development published by John Wiley & Sons Ltd, Child: care, health and development
The TRANSITION-Q 7
Figure 1. (a) Category probability curves for item ‘I see the doctor or nurse on my own during an appointment’ before rescoring. The x-axis represents the construct of self-management skills with higher scores increasing to the right. The y-axis shows the probability of endorsing the response categories, reading left to right: 0 (first curve) ‘never’, 1 (second curve) ‘sometimes’, 2 (third curve) ‘often’ and 3 (fourth curve) ‘always’. (b) The item ‘I see the doctor or nurse on my own during an appointment’ after rescoring. Here, the y-axis shows the probability of endorsing the response categories, reading left to right: 0 (first curve) ‘never’, 1 (second curve) ‘sometimes’ or ‘often’, 2 (third curve) ‘always’.
Table 3. Statistical indicators of fit, including fit residual and chi-square Item description
Item location
SE
Fit residual
d.f.
Chi-square
d.f.
P-value
I answer a doctor’s or nurse’s questions. I help to make decisions about my health. I am in charge of taking any medicine that I need. I talk to a doctor or nurse when I have health concerns. I look for an answer when I have a question about my health. I talk about my health condition to people when I need to. I ask the doctor or nurse questions. I speak to the doctor instead of my parent(s) speaking for me. I summarize my medical history when I am asked to. I contact a doctor when I need to. I see the doctor or nurse on my own during an appointment. I drop off or pick up my prescriptions when I need medicine. I travel on my own to a doctor’s appointment. I book my own doctor’s appointments.
−2.41 −1.21 −0.99 −0.84 −0.76 −0.74 −0.33 −0.28 −0.15 0.39 0.61 1.57 2.52 2.62
0.12 0.11 0.10 0.12 0.11 0.11 0.12 0.11 0.10 0.10 0.09 0.10 0.12 0.13
1.16 0.03 2.12 −0.85 −0.91 1.30 −0.61 −1.04 −1.75 −1.29 1.94 −0.02 −0.33 −1.40
300 310 309 310 311 310 309 309 311 310 309 309 311 310
4.56 4.77 16.13 6.57 5.92 4.92 2.64 8.21 5.74 5.47 2.65 5.32 6.66 8.62
5 5 5 5 5 5 5 5 5 5 5 5 5 5
0.47 0.44 0.01 0.26 0.31 0.43 0.76 0.15 0.33 0.36 0.75 0.39 0.25 0.13
d.f., degrees of freedom; SE, standard error.
Targeting Figure 2 shows the distribution of person measurements (top histogram) and item locations (bottom histogram) and provides evidence that our study sample lies inside the range in which the scale provides measurement. The scale thus defined a continuum for self-management skills and was well targeted to the sample of participants.
Stability DIF was not detected for gender or age after Bonferroni correction (results not shown but available by contacting the corresponding author).
PSI The PSI value was 0.82 indicating good reliability.
© 2014 The Authors. Child: Care, Health and Development published by John Wiley & Sons Ltd, Child: care, health and development
8
A.F. Klassen et al.
Figure 2. Person-item thresholds distribution. The x-axis represents the construct (self-management skills), with higher scores (more skills) increasing to the right. The y-axis shows the frequency of person measure locations (top histogram) and item locations (bottom histogram).
Dependency Residual correlations for 12/14 items were below 0.30, supporting local independence. Residual correlations for the two items that were marginally above 0.30 were sub-tested and found to have little impact on reliability, i.e. PSI was 0.80 after sub-testing. Overall model fit A non-significant chi-squared value was achieved (chi squared = 88.17, degrees of freedom = 70, P-value = 0.07), supporting data fit to the Rasch model.
CTT analysis results Flesch–Kincaid grade level for the 14 items ranged from 2.2 to 7.1 (average 4.4), with one item above grade 6. Data quality was high, with little item level missing data (range 0 to 3.6%). Raw scale total scores were computable for the 90.2% (i.e. participants who provided answers to all 14 items). Scaling assumptions and scale-to-sample targeting (see Table 4) were appropriate given the response pattern for a scale designed to measure a clinical hierarchy, e.g. the pattern of responses for floor/ceiling effect ranged from 1/48 (easiest answer to endorse) to 78/3 (hardest item to endorse). More specifically, the easiest skill for participants to endorse was ‘I answer a doctor’s or nurse’s question’. This item has the highest mean score (2.31) and largest ceiling effect. The most difficult item to endorse was ‘I book my own doctor’s appointments. This skill had the lowest mean score (0.35) with the smallest ceiling effect (3%).
Internal consistency reliability was high, with a Cronbach’s alpha of 0.85. For the TRT, 220 (65.3%) participants agreed to participate but only 47 completed a second copy of the scale online (response rate of 21.4%). The number of days between the test and re-test ranged from 14 to 54; 87.2%of re-tests were completed by day 21. The intra-class correlation coefficient for the 47 subjects was 0.90 and for those who responded by day 21 was 0.92. The mean scale score was lower for participants who were younger (Pearson correlation = 0.46; P < 0.01), and who needed assistance to complete the scale (mean = 52.7, SD = 12.4 versus mean = 43.3, SD = 13.3; P < 0.01 on t-test). Finally, the mean score was incrementally higher (P < 0.01 on analysis of variance) according to the strength of agreement to the item: ‘I am ready to transfer to adult health care’ (see Fig. 3).
Discussion It is recommended that adolescents with chronic health conditions should prepare for transition in early adolescence (American Academy of Pediatrics et al. 2002, 2011; Gorter et al. 2011; McManus et al. 2013). However, given the lack of validated transition tools available to date, it is not surprising that many transition programmes in Canada have incorporated ad hoc checklists into clinical practice. Ad hoc measures may pose reasonable questions and serve as a practical clinical guide, but we cannot be confident about the reliability (i.e. ability to
© 2014 The Authors. Child: Care, Health and Development published by John Wiley & Sons Ltd, Child: care, health and development
The TRANSITION-Q 9
Table 4. Traditional psychometric methods including data quality, scaling assumptions, targeting and reliability
Item description I answer a doctor’s or nurse’s questions. I help to make decisions about my health. I am in charge of taking any medicine that I need. I talk to a doctor or nurse when I have health concerns. I look for an answer when I have a question about my health. I talk about my health condition to people when I need to. I ask the doctor or nurse questions. I speak to the doctor instead of my parent(s) speaking for me. I summarize my medical history when I am asked to. I contact a doctor when I need to. I see the doctor or nurse on my own during an appointment. I drop off or pick up my prescriptions when I need medicine. I travel on my own to a doctor’s appointment. I book my own doctor’s appointments.
Data quality
Scaling assumptions
Missing data (%)
Possible score range
Actual score range
Mean score
SD
CITC
3.6 0.3 0.3 0.3
0–3 0–3 0–3 0–3
0–3 0–3 0–3 0–3
2.31 2.07 2.17 1.88
0.76 0.86 0.98 0.90
0.42 0.52 0.32 0.57
1/48 5/36 8/50 6/29
−0.67 −0.55 −0.87 −0.27
0
0–3
0–3
1.95
0.93
0.55
8/33
−0.50
0.3
0–3
0–3
1.86
0.92
0.42
7/29
−0.23
0.6 0.6
0–3 0–3
0–3 0–3
1.60 1.67
0.89 0.93
0.53 0.65
8/19 10/22
0.17 −0.08
0 0.3 0.6
0–3 0–3 0–3
0–3 0–3 0–3
1.69 1.40 1.22
1.06 1.10 1.09
0.62 0.57 0.43
17/28 27/21 33/18
−0.23 0.13 0.40
0.6
0–3
0–3
0.63
0.99
0.47
64/10
1.42
0.3 0.3
0–3 0–3
0–3 0–3
0.29 0.35
0.70 0.73
0.41 0.47
82/3 78/3
2.59 2.16
100
80
60
40
20
0 Definitely Disagree Somewhat Disagree
Somewhat Agree
Definitely Agree
Figure 3. Mean TRANSITION-Q score for participants by strength of agreement to the statement: I am ready to transfer to adult health care.
produce consistent and reproducible scores) or validity (i.e. ability to measure what is intended to be measured) of any measure that has not been formally developed or tested. The TRANSITION-Q represents a new psychometrically sound and clinically meaningful scale that can be used in transition programmes with adolescents starting at 12 years of age
Targeting Floor/ceiling (%)
Skewness
to measure and track the development of skills they need to acquire to manage their health and healthcare. The content of the scale was designed to include a range of skills that vary from those that even young adolescents should be able to do (e.g. answer a doctor’s or nurse’s questions) to skills that may require instruction or training (e.g. book a clinical appointment). The use of this scale in clinical practice by healthcare providers (e.g., with electronic data collection and real-time generation of patient reports showing the pattern of responses to the 14 items) would make it possible to identify patients’ strengths as well as areas for improvement so that these can be addressed (Greenhalgh & Meadows 1999; Marshall et al. 2006; Valderas et al. 2008). The scale can be used also in transition readiness research where studies are needed to document the outcomes of selfmanagement interventions; a recent systematic review of selfmanagement interventions for children and youth with disabilities found that only six studies of distinct programmes were published in a 30-year period providing only limited evidence that self-management interventions are effective (Lindsay et al. 2014). Self-management skills represent just one of an array of factors hypothesized to influence transition readiness in the Social Ecological Model of Adolescents and Young Adults Readiness for Transition (Schwartz et al. 2011). This model accounts for a range of objective modifiable subjective factors important in the process of transition for patients with
© 2014 The Authors. Child: Care, Health and Development published by John Wiley & Sons Ltd, Child: care, health and development
10
A.F. Klassen et al.
chronic illnesses (e.g. knowledge, skills, self-efficacy, beliefs/ expectations, goals, relationships, psychosocial functioning). This framework provides an evidence-based framework that could be used to guide the development and implementation of transition readiness research. It would be interesting to investigate the relationship between the specific skills measured by the TRANSITION-Q and skills in various life domains, as in the end, all these crucial life skills are required for adolescents to achieve their life goals and experience a successful transition. Our study has several limitations. While an important strength is the large sample size recruited from nine different subspecialty clinics, some health conditions in our sample were under-represented (e.g. asthma, cystic fibrosis) and some common health conditions were not included (e.g. arthritis). Research to examine the psychometric properties of the TRANSITION-Q in under-represented and additional patient groups will be required. Second, the study sample was recruited from a single institution, and we recognize that the inclusion of more hospitals may have increased the heterogeneity of the sample even further. Third, while the response rate was high and shows that adolescents are willing to complete a short questionnaire during the course of a hospital visit, the response to the additional TRT survey was low. It is possible adolescents approached in our study may not have been comfortable saying no when asked face-to-face to complete the TRT at home, or that an email with a link to an online survey is not an optimal method for data collection from adolescents. Finally, our scale measures a single construct, i.e. skills required to manage one’s health and healthcare. We recognize that the development of self-management skills outside of healthcare (e.g. education, work, daily life) are also important to transition success, as are other modifiable subjective variables (Schwartz et al. 2011) such as the concept of self-efficacy (i.e. a person’s belief in his or her ability to succeed in specific situations (Bandura 1977). As such, there is scope for future research to develop additional clinically meaningful scales to measure other important transition-related constructs. A new study to further test the TRANSITION-Q across six paediatric healthcare centres in Ontario for further implementation and validation is now in preparation. In conclusion, we have provided evidence for reliability and validity of the TRANSITION-Q. While there is a need for further psychometric work to be carried out to add to the evidence base for the use of this scale and the generalizability of its measurement properties, we argue that the clinical meaning of the scale’s scores can be best established when the TRANSITION-Q tool is used and evaluated broadly.
Key messages • Three recent systematic reviews identified a lack of methodologically sound transition readiness scales. • The TRANSITION-Q is a reliable and valid 14-item scale that measures self-management skills in health and healthcare in adolescents aged 12 to 18 years across a broad range of chronic health conditions. • The TRANSITION-Q was designed using a modern psychometric approach to rating scale development and validation, i.e. Rasch measurement theory analysis. • The TRANSITION-Q can be used in clinical practice by healthcare professionals to help adolescents develop the skills needed to take care of their health and healthcare needs as adults. • The TRANSITION-Q can be used in research studies to understand factors that influence transition readiness.
Funding Anne Klassen holds a Canadian Institutes of Health Research Mid-Career Award, which provides a research stipend that was used alongside matched funding from the Department of Pediatrics at McMaster University to fund this study. Jan Willem Gorter holds the Scotiabank Chair in Child Health Research (2013–2017).
Conflict of interests None of the authors has any conflicts of interest to declare.
References Aaronson, N., Alonso, J., Burnam, A., Lohr, K. N., Patrick, D. L., Perrin, E. & Stein, R. E. (2002) Assessing health status and quality-of-life instruments: attributes and review criteria. Quality of Life Research: An International Journal of Quality of Life Aspects of Treatment, Care and Rehabilitation, 11, 193–205. American Academy of Pediatrics, American Academy of Family Physicians & American College of Physicians-American Society of Internal Medicine (2002) A consensus statement on health care transitions for young adults with special health care needs. Pediatrics, 110, 1304–1306. American Academy of Pediatrics, American Academy of Family Physicians, American College of Physicians, Transitions Clinical Report Authoring Group, Cooley, W. C. & Sagerman, P. J. (2011) Supporting the health care transition from adolescence to adulthood in the medical home. Pediatrics, 128, 182–200.
© 2014 The Authors. Child: Care, Health and Development published by John Wiley & Sons Ltd, Child: care, health and development
The TRANSITION-Q 11
Andrich, D. (1982) An index of person separation in latent trait theory, the traditional KR20 index, and the Guttman scale response pattern. Education Research Perspectives, 9, 95–104. Andrich, D. (1988) Rasch Models for Measurement. Sage, Newbury Park, CA, USA. Andrich, D. (2004) Controversy and the Rasch model: a characteristic of incompatible paradigms? Medical Care, 42, I7–16. Andrich, D. & Sheridan, B. (1997–2011) RUMM2030. RUMM Laboratory, Perth, Australia. Bandura, A. (1977) Self-efficacy: towards a unifying theory of behavioral change. Psychological Review, 84, 191–215. Bevans, K. B., Riley, A. W., Moon, J. & Forrest, C. B. (2010) Conceptual and methodological advances in child-reported outcomes measurement. Expert Review of Pharmacoeconomics & Outcomes Research, 10, 385–386. Blum, R. W. (2002) Improving transition for adolescents with special health care needs from pediatric to adult-centered health care. Pediatrics, 110, 1301–1303. Canadian Paediatric Society, Adolescent Health Committee, [Principal authors: Kaufman, M. & Pinzon, J.] (2007) Transition to adult care for youth with special health care needs. Paediatrics and Child Health, 12, 785–788. Cronbach, L. (1951) Coefficient alpha and the internal structure of tests. Psychometrika, 16, 297–334. Flesch, R. (1948) A new readability yardstick. Journal of Applied Psychology, 32, 221–233. Giraudeau, B. & Mary, J. Y. (2001) Planning a reproducibility study: how many subjects and how many replicates per subject for an expected width of the 95 per cent confidence interval of the intraclass correlation coefficient. Statistics in Medicine, 20, 3205–3214. Gorter, J. W., Stewart, D. & Woodbury-Smith, M. (2011) Youth in transition: care, health and development. Child: Care, Health and Development, 37, 757–763. Grant, C. & Pan, J. (2011) A comparison of five transition programmes for youth with chronic illness in Canada. Child: Care, Health and Development, 37, 815–820. Greenhalgh, J. & Meadows, K. (1999) The effectiveness of the use of patient-based measures of health in routine practice in improving the process and outcomes of patient care: a literature review. Journal of Evaluation in Clinical Practice, 5, 401–416. Hagquist, C. & Andrich, D. (2004) Is the sense of coherence-instrument applicable on adolescents? A latent trait analysis using Rasch-modelling. Personality and Individual Differences, 36, 955–968. Hobart, J. & Cano, S. (2009) Improving the evaluation of therapeutic interventions in multiple sclerosis: the role of new psychometric methods. Health Technology Assessment, 13, 1–200. Klassen, A. F., Rosenberg-Yunger, Z. R., D’Agostino, N. M., Cano, S. J., Barr, R., Syed, I., Granek, L., Greenberg, M. L., Dix, D. & Nathan, P. C. (2014) The development of scales to measure childhood cancer survivors’ readiness for transition to long-term follow-up care as adults. Health Expectations, July 22 [Epub ahead of print].
Likert, R. (1932) A technique for the measurement of attitudes. Archives of Psychology, 140, 1–55. Lindsay, S., Kingsnorth, S., McDougall, C. & Keating, H. (2014) A systematic review of self-management interventions for children and youth with physical disabilities. Disability and Rehabilitation, 36, 276–288. Marais, I. & Andrich, D. (2008) Formalizing dimension and response violations of local independence in the unidimensional Rasch model. Journal of Applied Measurement, 9, 200–215. Marshall, S., Haywood, K. & Fitzpatrick, R. (2006) Impact of patient-reported outcome measures on routine practice: a structured review. Journal of Evaluation in Clinical Practice, 12, 559–568. McHorney, C. A., Haley, S. M. & Ware, J. E., Jr (1997) Evaluation of the MOS SF-36 Physical Functioning Scale (PF-10): II. Comparison of relative precision using Likert and Rasch scoring methods. Journal of Clinical Epidemiology, 50, 451–461. McManus, M. A., Pollack, L. R., Cooley, W. C., McAllister, J. W., Lotstein, D., Strickland, B. & Mann, M. Y. (2013) Current status of transition preparation among youth with special needs in the United States. Pediatrics, 131, 1090–1097. Nunnally, J. C. & Berstein, I. H. (1994) Psychometric Theory, 3rd edn. McGraw-Hill, New York, NY, USA. Paone, M. C., Wigle, M. & Saewyc, E. (2006) The ON TRAC model for transitional care of adolescents. Progress in Transplantation, 16, 291–302. Patrick, D. L., Burke, L. B., Gwaltney, C. J., Leidy, N. K., Martin, M. L., Molsen, E. & Ring, L. (2011a) Content validity – establishing and reporting the evidence in newly developed patient-reported outcomes (PRO) instruments for medical product evaluation: ISPOR PRO good research practices task force report: part 1 – eliciting concepts for a new PRO instrument. Value in Health, 14, 967–977. Patrick, D. L., Burke, L. B., Gwaltney, C. J., Leidy, N. K., Martin, M. L., Molsen, E. & Ring, L. (2011b) Content validity – establishing and reporting the evidence in newly developed patient-reported outcomes (PRO) instruments for medical product evaluation: ISPOR PRO Good Research Practices Task Force report: part 2-assessing respondent understanding. Value in Health, 14, 978–988. Provincial Council for Maternal and Child Health (2013) Transition to adult healthcare services work group report. Available at: http:// www.pcmch.on.ca/initiatives/transition-adult-healthcare-services (last accessed 24 April 2014). Rasch, G. (1960) Probabilistic Models for Some Intelligence and Attainment Tests. Danish Institute for Education Research, Copenhagen, Denmark. Rosen, D. S., Blum, R. W., Britto, M., Sawyer, S. M., Siegel, D. M. & Society for Adolescent Medicine (2003) Transition to adult health care for adolescents and young adults with chronic conditions: position paper of the Society for Adolescent Medicine. Journal of Adolescent Health, 33, 309–311. Sawicki, G. S., Lukens-Bull, K., Yin, X., Demars, N., Huang, I. C., Livingood, W., Reiss, J. & Wood, D. (2011) Measuring the
© 2014 The Authors. Child: Care, Health and Development published by John Wiley & Sons Ltd, Child: care, health and development
12
A.F. Klassen et al.
transition readiness of youth with special healthcare needs: validation of the TRAQ – Transition Readiness Assessment Questionnaire. Journal of Pediatric Psychology, 36, 160–171. Schwartz, L. A., Tuchman, L. K., Hobbie, W. L. & Ginsberg, J. P. (2011) A social-ecological model of readiness for transition to adult-oriented care for adolescents and young adults with chronic health conditions. Child: Care, Health and Development, 37, 883–895. Schwartz, L. A., Daniel, L. C., Brumley, L. D., Barakat, L. P., Wesley, K. M. & Tuchman, L. K. (2014) Measures of readiness to transition to adult health care for youth with chronic physical health conditions: a systematic review and recommendations for measurement testing and development. Journal of Pediatric Psychology, 39, 588–601. Stinson, J., Kohut, S. A., Spiegel, L., White, M., Gill, N., Colbourne, G., Sigurdson, S., Duffy, K. W., Tucker, L., Stringer, E., Hazel, B., Hochman, J., Reiss, J. & Kaufman, M. (2013) A systematic review of transition readiness and transfer satisfaction measures for adolescents with chronic illness. International Journal of Adolescent Medicine and Health, 6, 1–16. [Epub ahead of print]. Terwee, C. B., Bot, S. D., de Boer, M. R., van der Windt, D. A., Knol, D. L., Dekker, J., Bouter, L. M. & de Vet, H. C. (2007) Quality criteria were proposed for measurement properties of health status questionnaires. Journal of Clinical Epidemiology, 60, 34–42. The Hospital for Sick Children (2011) Readiness checklist. Available at: http://www.sickkids.ca/Good2Go/Transition-Interventions
-Tools/Readiness-checklists/index.html (last accessed 9 December 2012). US Department of Health and Human Services (2009) Guidance for industry: patient reported outcome measures: use in medical product development to support labeling claims. Available at: http:// www.ispor.org/workpaper/FDA%20PRO%20Guidance.pdf (last accessed 24 April 2014). Valderas, J. M., Kotzeva, A., Espallargues, M., Guyatt, G., Ferrans, C. E., Halyard, M. Y., Revicki, D. A., Symonds, T., Parada, A. & Alonso, J. (2008) The impact of measuring patient-reported outcomes in clinical practice: a systematic review of the literature. Quality of Life Research: An International Journal of Quality of Life Aspects of Treatment, Care and Rehabilitation, 17, 179–193. Ware, J. E., Jr, Harris, W. J., Gandek, B., Rogers, B. W. & Reese, P. R. (1996) MAP-R for Windows: Multitrait/Multi-Item Analysis Program – Revised User’s Guide. Health Assessment Laboratory, Boston, MA, USA. Wright, B. & Masters, G. (1982) Rating Scale Analysis: Rasch Measurement. MESA, Chicago, IL, USA. Wright, B. D. & Stone, M. H. (1979) Best Test Design: Rasch Measurement, pp. 1–17. MESA, Chicago, IL, USA. Zhang, L. F., Ho, J. S. & Kennedy, S. E. (2014) A systematic review of the psychometric properties of transition readiness assessment tools in adolescents with chronic disease. BMC Pediatrics, 14, 4.
© 2014 The Authors. Child: Care, Health and Development published by John Wiley & Sons Ltd, Child: care, health and development
