Article
A preliminary evaluation of the Visual CARE Measure for use by Allied Health Professionals with children and their parents
Journal of Child Health Care 1–13 ª The Author(s) 2014 Reprints and permission: sagepub.co.uk/journalsPermissions.nav DOI: 10.1177/1367493514551307 chc.sagepub.com
Morag A Place1, Joan Murphy1, Edward AS Duncan2, Jane M Reid3 and Stewart W Mercer4
Abstract The Consultation and Relational Empathy (CARE) Measure (Mercer et al., 2004) is a patientrated experience measure of practitioner empathy, developed and validated within adult health services. This study reports the feasibility, acceptability, reliability and validity of three adapted versions of the original CARE measure for the paediatric setting, namely the Visual CARE Measure 5Q, 10Q and 10Q Parent (also known as the Paediatric CARE Measure). Three hundred and sixty-nine participants (N ¼ 149 children (40%) and N ¼ 220 parents (60%)) completed the measure following consultation with an Allied Health Professional (AHP). AHPs felt it was feasible to use the measure in routine practice and the majority of children and parents found the measure easy to understand (98%) and complete (98%). Internal reliability (Cronbach’s a) was .746 for the 5Q, .926 for the 10Q and .963 for the 10Q parent. Few participants used the ‘not applicable’ response (N ¼ 28 (8%)), suggesting high content validity. AHPs found the measures relevant (95%) and useful (90%) and reported that they were likely to use them again (96%). The Visual CARE Measure shows promise as a useful tool to enable children and their parents to give their views. Further research on the tool’s reliability and validity is required.
1
Talking Mats Centre, Scotland NMAHP Research Unit, University of Stirling, Scotland 3 NHSScotland, Scotland 4 General Practice and Primary Care, Institute of Health and Wellbeing, University of Glasgow, Scotland 2
Corresponding author: Joan Murphy, Talking Mats Centre, Stirling University Innovation Park, 2 Beta Centre, Stirling FK9 4NF, Scotland. Email: [email protected]
Downloaded from chc.sagepub.com at University of Otago Library on September 28, 2015
2
Journal of Child Health Care
Keywords Children’s participation, evidence-based practice, professional development, quality of care, therapeutic relationships
Background Health-care services are increasingly focused on providing patient-centred care and part of that process involves encouraging feedback on the service provided (The Scottish Government, 2010, 2012; UK Government, 2012). However, seeking the views of children is complex. Questionnaires designed for adults will require modification and psychometric testing prior to use in paediatric services. Children (depending on age and ability) may have difficulty coping with the demands of responding to a questionnaire survey (comprehending the question, recalling the information required to respond, making a judgement and communicating their answers) (Bell, 2007). However, children can give their views on a variety of subjects, including their own health care, if given appropriate tools and support (Borgers et al., 2000; Hutchfield and Coren, 2011; Owen et al., 2004; Robinson, 2010; Taylor et al., 2010). This is important as these views can differ from proxy views (views provided by others on behalf of the child) (Scott, 2000). In addition, it is important for practitioners that any paediatric measure is not only relevant but also concise and easy to administer. The Consultation and Relational Empathy (CARE) Measure (Mercer et al., 2004) is a validated and widely used 10-item patient-rated experience questionnaire that allows health professionals to gain a patient’s perspective on the level of empathy shown by them during a clinical consultation. Although some have questioned the desirability of empathy in health care (Macnaughton, 2009), empathy has been shown to be an important element of the clinical encounter from both the patients’ and health-care professionals’ perspectives (Mercer and Reynolds, 2002) and influences patient enablement (Mercer et al., 2012) and some health outcomes (Price et al., 2006; Verheul et al., 2010). Neumann et al. (2009) have proposed a model to explain how the links between empathy in the physician–patient relationship might improve therapeutic outcomes. Patients are more likely to give more detailed information about their symptoms and concerns, resulting in a more accurate diagnosis. The empathic physician is able to have a better understanding of an individual patient’s needs. Consequently, empathic physicians can reciprocate by means of specific therapies and provide appropriate illness-related information, allowing patient participation whilst doing so. This leads to improvements in patient satisfaction, patient enablement and patient compliance with proposed therapies. Originally designed for use within general practice, the CARE Measure has been validated with a range of other health professionals in various clinical settings (Duncan et al., in preparation; Mercer and Murphy, 2008; Mercer et al., 2008). It was, however, designed for adults and, in its original structure, may not be appropriate for use in paediatric services. Phase 1 of this study (Place et al., 2012) reported preliminary interview and observational data on the acceptability and usability of two adapted versions of the original CARE measure (Visual CARE Measure 10Q and 10Q Parent). Adaptations included use of a visual ‘faces’ scale and the addition of symbols to enhance understanding of each question. Feedback was given by children (N ¼ 25) and parents (N ¼ 13) on the ease of completion, ease of understanding and importance of the questions. Observational data were also collected on how well participants coped with the measure’s vocabulary, language complexity, symbols, length and layout. This information was then used to revise the Visual CARE Measure (resulting in three versions, namely 5Q, 10Q and 10Q Parent). The 5Q was developed for both younger children (aged 7–11 years) and those with 2 Downloaded from chc.sagepub.com at University of Otago Library on September 28, 2015
Place et al.
3
additional support needs (ASN), following results showing that these groups required a measure with fewer questions and further simplified language. This article reports on the feasibility, acceptability, preliminary reliability and validity of these latest versions of the Visual CARE Measure (5Q, 10Q and 10Q Parent).
Method Ethics National Health Service (NHS) ethics advice was sought, and this project was classified as service evaluation and did not require specific ethical approvals. The research was conducted in keeping with good practice, and consent was discussed with participants and was understood to have been granted if participants completed the forms. All data were anonymous.
Context This study was conducted with Allied Health Professionals (AHPs) who routinely work with children and their parents in clinical and educational settings. The term ‘children’ has been used throughout to refer to minors of up to 18 years and should therefore be understood to include young people. In addition, the term ‘parent’ has been used to refer to parents, carers or guardians.
Participants Allied Health Professionals. Professionals from six disciplines working with children in three NHS health boards in Scotland were recruited (see Table 1). Children and parents. Three groups were recruited – children in mainstream primary or secondary education (7–18 years); children in special needs primary or secondary education (7–18 years); and parents of young children (0–6 years). Children or parents were excluded from the study if (i) they had a profound cognitive or language impairment, (ii) they had a severe visual impairment or (iii) they were too unwell.
Materials AHP information sheet. The AHP information sheet detailed the background to the project, the project procedure, exclusion criteria and what was expected of AHPs. This was sent out prior to meeting with AHPs. Visual CARE Measure project packs. There were three separate packs as follows: (1) 5Q – for children aged 7–11 at mainstream primary school or any child attending special school (a school providing education to children with ASN); (2) 10Q – for young people aged 11–18 at mainstream secondary school; and (3) 10Q Parent – for parents of children aged 0–6 or parents of children who met the exclusion criteria and therefore could not complete the measure themselves. The 10Q (young people) and 10Q Parent measures both have 10 questions, a visual 5-point scale and symbols to support understanding of each question. The only difference between the two 10Q measures is in the wording of questions to reflect who is completing the measure. The measures use 3 Downloaded from chc.sagepub.com at University of Otago Library on September 28, 2015
4
Journal of Child Health Care
Table 1. AHP details. Allied Health Profession SLT OT Physiotherapy Dietetics Orthoptics Podiatry Total
No. of No. of AHPs who AHPs signed up participated 52 45 40 9 6 2 154
No. of completed measures returned
38 (73%) 161 29 (64%) 107 19 (48%) 70 1 (11%) 2 4 (67%) 10 2 (100%) 15 93 (60%) 365 (þ4 missing AHP details ¼ 369)
Mean no. of measures returned per AHP
No. of completed AHP online surveys returned
4.24 3.69 3.68 2 3.56 7.5 3.97
42 (81%) 28 (62%) 21 (53%) 1 (11%) 4 (67%) 2 (100%) 98 (þ1 preferred not to state profession)
SLT: Speech and Language Therapy; OT: Occupational Therapy; AHPs: Allied Health Professionals.
visual supports in order that they are all consistent and the results are easily comparable. The 5Q has five questions each supported by a symbol and less complex vocabulary than the 10Q measures (see Figure 1 for extract from the Visual CARE Measure). Each pack also included a tailored information sheet, evaluation form and an A5 addressed envelope. Evaluation forms. The evaluation form asked participants, both adults and children, to rate, on a 4-point visual scale, how easy or difficult they found the Visual CARE Measure to complete and understand and how much support they required to complete it. It also requested information on age, gender, cognitive or language difficulties, language used at home, length, purpose and location of appointment. Space was provided for participants to give written comments. AHP online survey. On completion of the project, AHPs were asked to give their views on how easy the measure was to use in their day-to-day work, how useful it was, how relevant the items were, whether they would use it in the future and whether they had any positive or negative feedback on the measure.
Procedure AHPs in three NHS Health Boards in Scotland were allocated a unique identification number to enable data analysis and to keep track of numbers returned. Each AHP was asked to give the measure to children or parents who they would be seeing as part of their normal caseload for an initial assessment or review appointment. Participants were recruited consecutively. It was emphasised to AHPs that it was useful for the researchers to gather information and feedback from participants who found the measure challenging to complete as well as those who found it easy. Gathering data from participants with a wide range of abilities allowed recommendations to be made regarding the likely age and cognitive/language ability of children and parents who can complete the measure. Participants who agreed to take part were given the Visual CARE Measure pack immediately following the consultation and were asked to spend a few minutes completing the forms. The 4 Downloaded from chc.sagepub.com at University of Otago Library on September 28, 2015
Place et al.
5
Figure 1. Extract from Visual CARE Measure 5Q. CARE: Consultation and Relational Empathy.
participant then placed all completed forms into the addressed envelope, sealed it and handed it over at a designated point. The AHP was not present during the completion of the measure. If the child or parent required support to complete the forms, someone independent from the AHP provided support (usually a parent or support worker). At the end of the visit, the AHP collected all sealed envelopes and returned them in batches to the research team. In the home setting, the participant completed the forms at home once the AHP had left and posted the envelope to the researchers.
Data analysis All quantitative data were analysed using SPSS version 18. Qualitative data were collected in two ways. Data from participants came in the form of written comments on the evaluation forms, 5 Downloaded from chc.sagepub.com at University of Otago Library on September 28, 2015
6
Journal of Child Health Care
completed either by the participants themselves for the 10Q measure or by the supporter for the 5Q measure. The second source of qualitative data was from the online survey of AHPs on their experiences of using the measure. All qualitative data were analysed thematically. Feasibility and acceptability. Analysis focused specifically on the acceptability and usability of the measure for children, parents and AHPs. Descriptive analysis was used for the Likert ratings given. Counts of frequency of responses and median values were calculated. Any differences between the three measures were calculated using 2 with a significance level set at p ¼ .05. Counts were calculated for any incomplete forms returned by participants when completing the measure as a further indication of feasibility. Reliability. Analysis of the internal consistency of the three measures was done using Cronbach’s a. Face and content validity. Face and content validity was assessed based on the counts of notapplicable responses, AHPs’ ratings regarding the relevance and usefulness of the measure, whether they would use the measure again and qualitative written feedback.
Results Allied Health Professionals A total of 154 AHPs participated in the study and 369 completed Visual CARE Measures were returned (see Table 1).
Respondents Of the measures returned, 149 (40%) were from children and 220 (60%) were from parents. AHPs reported that only 11 patients refused to participate. See Table 2 for further details on the children and parents who completed the measures.
Feasibility and acceptability – Children’s and parent’s views Ease of understanding and completion. Overall ratings showed that the majority of participants found the Visual CARE Measure both easy to understand (98%) and easy to complete (98%; see Table 3). A further measure of feasibility was an examination of how many of the completed forms had missing values. There were low numbers of missing values (5Q: N ¼ 1 (1%); 10Q: N ¼ 4 (5%); 10Q Parent: N ¼ 12 (6%); see Table 5) with most being attributed to participants not completing the reverse page of both 10Q measures. Qualitative feedback indicated that the Visual CARE Measure was highly acceptable to children and parents, clear, easy to complete, easy to understand, simple, straightforward and the symbols were useful and appropriate for illustrating the questions. Support. Some support was required to complete the measure and to understand and read the scale and questions. 71% of children completing the 5Q measure, 32% completing the 10Q and 11% of parents required some support to understand and complete the measure (see Figure 2). The difference in support required was statistically significant using 2 analysis ( 2 ¼ 78.693, df ¼ 2, p < .001). 6 Downloaded from chc.sagepub.com at University of Otago Library on September 28, 2015
Place et al.
7
Table 2. Children’s and parent’s details. Age category of child Children 7–8: N ¼ 29 (27%) 9–10: N ¼ 29 (27%) 11–12: N ¼ 28 (26%) 13–14: N ¼ 14 (13%) 15–18: N ¼ 7 (7%) Mean age: 10.43 Range: 7–18 Total 107/149 Parents 0–2: N ¼ 58 (30%) 3–4: N ¼ 85 (44%) 5–6: N ¼ 50 (26%) Mean age: 3.37 Range: 0–6 Total
193/220
Gender
Education
Male: N ¼ 71 (65%) Female: N ¼ 38 (35%)
Mainstream N ¼ 139 (94%) Special school N ¼ 10 (6%)
Language/cognitive impairment
Language used at home
No: N ¼ 81 (70%) Yes: N ¼ 34 (30%)
English: N ¼ 116 (97%) Other/bilingual: N ¼ 3 (3%)
109/149 149/149 Male: N ¼ 75 (41%) N/A Female: N ¼ 109 (59%)
115/149 119/149 No: N ¼ 180 (91%) English: N ¼ 174 Yes: N ¼ 17 (9%) (93%) Other / bilingual: N ¼ 13 (7%)
184/220
197/220
187/220
N/A: not applicable.
Table 3. Ease of understanding and completion.
Measure
Very difficult
Quite difficult
10Q 10Q Parent 5Q (child rating) 5Q (Supporters’ rating) 10Q 10Q Parent 5Q (child rating) 5Q (supporters’ rating)
0 0 1 (1%) 0 0 0 0 1 (1%)
0 1 6 3 1 2 3 3
(1%) (7%) (4%) (2%) (1%) (3%) (4%)
Quite easy 15 52 26 29 10 52 27 34
(31%) (24.5%) (30%) (38%) (20%) (25%) (32%) (45%)
Very easy 34 158 53 44 38 156 55 37
(69%) (74.5%) (62%) (58%) (78%) (74%) (65%) (49%)
Total 49 (þ3 missing) 211 (þ6 missing) 86 (þ1 missing) 76 (þ11 missing) 49 (þ3 missing) 210 (þ7 missing) 85 (þ2 missing) 75 (þ12 missing)
Median rating for ease of understanding 4 4 4 4 4 4 4 3
(Very easy) (Very easy) (Very easy) (Very easy) (Very easy) (Very easy) (Very easy) (Quite easy)
Feasibility and acceptability – AHP views Among the AHPs who responded to the online survey, 84% said it was quite easy or very easy to administer the measure in their day-to-day work. Qualitative feedback from AHPs described their experiences of using the measure. Handing out the measure. The majority of AHPs reported that it was very easy to hand out the measure and explained it was ‘easy to use in clinic setting and easy to explain to clients’ (AHP 86, SLT). Some commented that having just attended a long appointment, they felt some families could be reluctant to spend any extra time completing the measure. By the end of an assessment most parents just want to get away with restless small children. (AHP 13, SLT) 7 Downloaded from chc.sagepub.com at University of Otago Library on September 28, 2015
8
Journal of Child Health Care
Table 4. Cronbach a analyses of 5Q, 10Q and 10Q Parent. Items removed for Cronbach asensitivity analyses
5Q
10Q
10Q (Parent)
No items removed Item 1 removed Item 2 removed Item 3 removed Item 4 removed Item 5 removed Item 6 removed Item 7 removed Item 8 removed Item 9 removed Item 10 removed
.746 .659 .750 .684 .701 .715 – – – – –
.926 .921 .927 .916 .917 .913 .918 .919 .915 .917 .920
.963 .961 .959 .959 .962 .957 .958 .958 .959 .957 .961
Participants’ responses. Many AHPs reported that both children and parents were happy to give feedback. Children appeared to be pleased that they were being given the opportunity to have their say. (AHP 85, OT) Parents appreciated the opportunity to feedback on their experience. (AHP 48, Physio)
Ease of understanding and completion. Many AHPs felt that children and parents found the Visual CARE Measure quick and easy to understand and complete, for example, ‘parents and children reported that it was easy to use’ (AHP 75, OT). Most AHPs felt the measure was aimed at just the right level with not too much language used. Participants with ASN. A few AHPs reflected that the measure may not be suitable for some children. I was unable to use it as the questionnaire had not been modified for the client group that I work with – Visual Impairment. (AHP 87, OT) I don’t think it would be appropriate for the majority of our school case load which is children with complex needs. (AHP 17, Physio)
Reliability Single administration reliability estimate. Internal consistency using Cronbach’s a for the 10Q was .926. Removal of any of the individual items reduced the internal consistency except for item 2; removal of this item slightly increased the internal consistency of the measure. The Cronbach’s a for the 10Q (Parent) was .963 (see Table 4) with removal of any of the questions weakening the internal reliability. The internal consistency of the 5Q using Cronbach’s a was .746. Removal of items 1, 3, 4 or 5 reduced the internal consistency, whereas removal of item 2 slightly increased the internal consistency of the measure; the difference was very slight (Table 4). Maximum possible score. On the 5Q measure, 28% of participants scored the maximum score of 25, on the 10Q measure, 35% of participants scored the maximum score of 50 and on the 10Q Parent, 54% scored the maximum score of 50 (see Table 5). 8 Downloaded from chc.sagepub.com at University of Otago Library on September 28, 2015
Place et al.
9
Table 5. Maximum score, skew, kurtosis and N/A responses. 5Q
10Q
10Q Parent
87 (83 þ 4 N/A 53 (43 þ 10 N/A 217 (185 þ 32 N/A or missing) or missing) or missing) Mean (SD) 22.2 (2.84) 45.7 (5.1) 46.3 (5.6) Min–Max 13–25 (max 28–50 (max 25–50 (max score 25) score 50) score 50) % of Respondents with max score 28% 35% 54% Skew –1.158 –1.456 –1.595 Kurtosis .902 2.277 1.829 Cronbach’s a .746 .926 .963 % N/A responses N ¼ 3 (3%) N ¼ 6 (11%) N ¼ 19 (9%) % >2 N/A N ¼ 0 (0%) N ¼ 1 (2%) N ¼ 2 (
A preliminary evaluation of the Visual CARE Measure for use by Allied Health Professionals with children and their parents
Journal of Child Health Care 1–13 ª The Author(s) 2014 Reprints and permission: sagepub.co.uk/journalsPermissions.nav DOI: 10.1177/1367493514551307 chc.sagepub.com
Morag A Place1, Joan Murphy1, Edward AS Duncan2, Jane M Reid3 and Stewart W Mercer4
Abstract The Consultation and Relational Empathy (CARE) Measure (Mercer et al., 2004) is a patientrated experience measure of practitioner empathy, developed and validated within adult health services. This study reports the feasibility, acceptability, reliability and validity of three adapted versions of the original CARE measure for the paediatric setting, namely the Visual CARE Measure 5Q, 10Q and 10Q Parent (also known as the Paediatric CARE Measure). Three hundred and sixty-nine participants (N ¼ 149 children (40%) and N ¼ 220 parents (60%)) completed the measure following consultation with an Allied Health Professional (AHP). AHPs felt it was feasible to use the measure in routine practice and the majority of children and parents found the measure easy to understand (98%) and complete (98%). Internal reliability (Cronbach’s a) was .746 for the 5Q, .926 for the 10Q and .963 for the 10Q parent. Few participants used the ‘not applicable’ response (N ¼ 28 (8%)), suggesting high content validity. AHPs found the measures relevant (95%) and useful (90%) and reported that they were likely to use them again (96%). The Visual CARE Measure shows promise as a useful tool to enable children and their parents to give their views. Further research on the tool’s reliability and validity is required.
1
Talking Mats Centre, Scotland NMAHP Research Unit, University of Stirling, Scotland 3 NHSScotland, Scotland 4 General Practice and Primary Care, Institute of Health and Wellbeing, University of Glasgow, Scotland 2
Corresponding author: Joan Murphy, Talking Mats Centre, Stirling University Innovation Park, 2 Beta Centre, Stirling FK9 4NF, Scotland. Email: [email protected]
Downloaded from chc.sagepub.com at University of Otago Library on September 28, 2015
2
Journal of Child Health Care
Keywords Children’s participation, evidence-based practice, professional development, quality of care, therapeutic relationships
Background Health-care services are increasingly focused on providing patient-centred care and part of that process involves encouraging feedback on the service provided (The Scottish Government, 2010, 2012; UK Government, 2012). However, seeking the views of children is complex. Questionnaires designed for adults will require modification and psychometric testing prior to use in paediatric services. Children (depending on age and ability) may have difficulty coping with the demands of responding to a questionnaire survey (comprehending the question, recalling the information required to respond, making a judgement and communicating their answers) (Bell, 2007). However, children can give their views on a variety of subjects, including their own health care, if given appropriate tools and support (Borgers et al., 2000; Hutchfield and Coren, 2011; Owen et al., 2004; Robinson, 2010; Taylor et al., 2010). This is important as these views can differ from proxy views (views provided by others on behalf of the child) (Scott, 2000). In addition, it is important for practitioners that any paediatric measure is not only relevant but also concise and easy to administer. The Consultation and Relational Empathy (CARE) Measure (Mercer et al., 2004) is a validated and widely used 10-item patient-rated experience questionnaire that allows health professionals to gain a patient’s perspective on the level of empathy shown by them during a clinical consultation. Although some have questioned the desirability of empathy in health care (Macnaughton, 2009), empathy has been shown to be an important element of the clinical encounter from both the patients’ and health-care professionals’ perspectives (Mercer and Reynolds, 2002) and influences patient enablement (Mercer et al., 2012) and some health outcomes (Price et al., 2006; Verheul et al., 2010). Neumann et al. (2009) have proposed a model to explain how the links between empathy in the physician–patient relationship might improve therapeutic outcomes. Patients are more likely to give more detailed information about their symptoms and concerns, resulting in a more accurate diagnosis. The empathic physician is able to have a better understanding of an individual patient’s needs. Consequently, empathic physicians can reciprocate by means of specific therapies and provide appropriate illness-related information, allowing patient participation whilst doing so. This leads to improvements in patient satisfaction, patient enablement and patient compliance with proposed therapies. Originally designed for use within general practice, the CARE Measure has been validated with a range of other health professionals in various clinical settings (Duncan et al., in preparation; Mercer and Murphy, 2008; Mercer et al., 2008). It was, however, designed for adults and, in its original structure, may not be appropriate for use in paediatric services. Phase 1 of this study (Place et al., 2012) reported preliminary interview and observational data on the acceptability and usability of two adapted versions of the original CARE measure (Visual CARE Measure 10Q and 10Q Parent). Adaptations included use of a visual ‘faces’ scale and the addition of symbols to enhance understanding of each question. Feedback was given by children (N ¼ 25) and parents (N ¼ 13) on the ease of completion, ease of understanding and importance of the questions. Observational data were also collected on how well participants coped with the measure’s vocabulary, language complexity, symbols, length and layout. This information was then used to revise the Visual CARE Measure (resulting in three versions, namely 5Q, 10Q and 10Q Parent). The 5Q was developed for both younger children (aged 7–11 years) and those with 2 Downloaded from chc.sagepub.com at University of Otago Library on September 28, 2015
Place et al.
3
additional support needs (ASN), following results showing that these groups required a measure with fewer questions and further simplified language. This article reports on the feasibility, acceptability, preliminary reliability and validity of these latest versions of the Visual CARE Measure (5Q, 10Q and 10Q Parent).
Method Ethics National Health Service (NHS) ethics advice was sought, and this project was classified as service evaluation and did not require specific ethical approvals. The research was conducted in keeping with good practice, and consent was discussed with participants and was understood to have been granted if participants completed the forms. All data were anonymous.
Context This study was conducted with Allied Health Professionals (AHPs) who routinely work with children and their parents in clinical and educational settings. The term ‘children’ has been used throughout to refer to minors of up to 18 years and should therefore be understood to include young people. In addition, the term ‘parent’ has been used to refer to parents, carers or guardians.
Participants Allied Health Professionals. Professionals from six disciplines working with children in three NHS health boards in Scotland were recruited (see Table 1). Children and parents. Three groups were recruited – children in mainstream primary or secondary education (7–18 years); children in special needs primary or secondary education (7–18 years); and parents of young children (0–6 years). Children or parents were excluded from the study if (i) they had a profound cognitive or language impairment, (ii) they had a severe visual impairment or (iii) they were too unwell.
Materials AHP information sheet. The AHP information sheet detailed the background to the project, the project procedure, exclusion criteria and what was expected of AHPs. This was sent out prior to meeting with AHPs. Visual CARE Measure project packs. There were three separate packs as follows: (1) 5Q – for children aged 7–11 at mainstream primary school or any child attending special school (a school providing education to children with ASN); (2) 10Q – for young people aged 11–18 at mainstream secondary school; and (3) 10Q Parent – for parents of children aged 0–6 or parents of children who met the exclusion criteria and therefore could not complete the measure themselves. The 10Q (young people) and 10Q Parent measures both have 10 questions, a visual 5-point scale and symbols to support understanding of each question. The only difference between the two 10Q measures is in the wording of questions to reflect who is completing the measure. The measures use 3 Downloaded from chc.sagepub.com at University of Otago Library on September 28, 2015
4
Journal of Child Health Care
Table 1. AHP details. Allied Health Profession SLT OT Physiotherapy Dietetics Orthoptics Podiatry Total
No. of No. of AHPs who AHPs signed up participated 52 45 40 9 6 2 154
No. of completed measures returned
38 (73%) 161 29 (64%) 107 19 (48%) 70 1 (11%) 2 4 (67%) 10 2 (100%) 15 93 (60%) 365 (þ4 missing AHP details ¼ 369)
Mean no. of measures returned per AHP
No. of completed AHP online surveys returned
4.24 3.69 3.68 2 3.56 7.5 3.97
42 (81%) 28 (62%) 21 (53%) 1 (11%) 4 (67%) 2 (100%) 98 (þ1 preferred not to state profession)
SLT: Speech and Language Therapy; OT: Occupational Therapy; AHPs: Allied Health Professionals.
visual supports in order that they are all consistent and the results are easily comparable. The 5Q has five questions each supported by a symbol and less complex vocabulary than the 10Q measures (see Figure 1 for extract from the Visual CARE Measure). Each pack also included a tailored information sheet, evaluation form and an A5 addressed envelope. Evaluation forms. The evaluation form asked participants, both adults and children, to rate, on a 4-point visual scale, how easy or difficult they found the Visual CARE Measure to complete and understand and how much support they required to complete it. It also requested information on age, gender, cognitive or language difficulties, language used at home, length, purpose and location of appointment. Space was provided for participants to give written comments. AHP online survey. On completion of the project, AHPs were asked to give their views on how easy the measure was to use in their day-to-day work, how useful it was, how relevant the items were, whether they would use it in the future and whether they had any positive or negative feedback on the measure.
Procedure AHPs in three NHS Health Boards in Scotland were allocated a unique identification number to enable data analysis and to keep track of numbers returned. Each AHP was asked to give the measure to children or parents who they would be seeing as part of their normal caseload for an initial assessment or review appointment. Participants were recruited consecutively. It was emphasised to AHPs that it was useful for the researchers to gather information and feedback from participants who found the measure challenging to complete as well as those who found it easy. Gathering data from participants with a wide range of abilities allowed recommendations to be made regarding the likely age and cognitive/language ability of children and parents who can complete the measure. Participants who agreed to take part were given the Visual CARE Measure pack immediately following the consultation and were asked to spend a few minutes completing the forms. The 4 Downloaded from chc.sagepub.com at University of Otago Library on September 28, 2015
Place et al.
5
Figure 1. Extract from Visual CARE Measure 5Q. CARE: Consultation and Relational Empathy.
participant then placed all completed forms into the addressed envelope, sealed it and handed it over at a designated point. The AHP was not present during the completion of the measure. If the child or parent required support to complete the forms, someone independent from the AHP provided support (usually a parent or support worker). At the end of the visit, the AHP collected all sealed envelopes and returned them in batches to the research team. In the home setting, the participant completed the forms at home once the AHP had left and posted the envelope to the researchers.
Data analysis All quantitative data were analysed using SPSS version 18. Qualitative data were collected in two ways. Data from participants came in the form of written comments on the evaluation forms, 5 Downloaded from chc.sagepub.com at University of Otago Library on September 28, 2015
6
Journal of Child Health Care
completed either by the participants themselves for the 10Q measure or by the supporter for the 5Q measure. The second source of qualitative data was from the online survey of AHPs on their experiences of using the measure. All qualitative data were analysed thematically. Feasibility and acceptability. Analysis focused specifically on the acceptability and usability of the measure for children, parents and AHPs. Descriptive analysis was used for the Likert ratings given. Counts of frequency of responses and median values were calculated. Any differences between the three measures were calculated using 2 with a significance level set at p ¼ .05. Counts were calculated for any incomplete forms returned by participants when completing the measure as a further indication of feasibility. Reliability. Analysis of the internal consistency of the three measures was done using Cronbach’s a. Face and content validity. Face and content validity was assessed based on the counts of notapplicable responses, AHPs’ ratings regarding the relevance and usefulness of the measure, whether they would use the measure again and qualitative written feedback.
Results Allied Health Professionals A total of 154 AHPs participated in the study and 369 completed Visual CARE Measures were returned (see Table 1).
Respondents Of the measures returned, 149 (40%) were from children and 220 (60%) were from parents. AHPs reported that only 11 patients refused to participate. See Table 2 for further details on the children and parents who completed the measures.
Feasibility and acceptability – Children’s and parent’s views Ease of understanding and completion. Overall ratings showed that the majority of participants found the Visual CARE Measure both easy to understand (98%) and easy to complete (98%; see Table 3). A further measure of feasibility was an examination of how many of the completed forms had missing values. There were low numbers of missing values (5Q: N ¼ 1 (1%); 10Q: N ¼ 4 (5%); 10Q Parent: N ¼ 12 (6%); see Table 5) with most being attributed to participants not completing the reverse page of both 10Q measures. Qualitative feedback indicated that the Visual CARE Measure was highly acceptable to children and parents, clear, easy to complete, easy to understand, simple, straightforward and the symbols were useful and appropriate for illustrating the questions. Support. Some support was required to complete the measure and to understand and read the scale and questions. 71% of children completing the 5Q measure, 32% completing the 10Q and 11% of parents required some support to understand and complete the measure (see Figure 2). The difference in support required was statistically significant using 2 analysis ( 2 ¼ 78.693, df ¼ 2, p < .001). 6 Downloaded from chc.sagepub.com at University of Otago Library on September 28, 2015
Place et al.
7
Table 2. Children’s and parent’s details. Age category of child Children 7–8: N ¼ 29 (27%) 9–10: N ¼ 29 (27%) 11–12: N ¼ 28 (26%) 13–14: N ¼ 14 (13%) 15–18: N ¼ 7 (7%) Mean age: 10.43 Range: 7–18 Total 107/149 Parents 0–2: N ¼ 58 (30%) 3–4: N ¼ 85 (44%) 5–6: N ¼ 50 (26%) Mean age: 3.37 Range: 0–6 Total
193/220
Gender
Education
Male: N ¼ 71 (65%) Female: N ¼ 38 (35%)
Mainstream N ¼ 139 (94%) Special school N ¼ 10 (6%)
Language/cognitive impairment
Language used at home
No: N ¼ 81 (70%) Yes: N ¼ 34 (30%)
English: N ¼ 116 (97%) Other/bilingual: N ¼ 3 (3%)
109/149 149/149 Male: N ¼ 75 (41%) N/A Female: N ¼ 109 (59%)
115/149 119/149 No: N ¼ 180 (91%) English: N ¼ 174 Yes: N ¼ 17 (9%) (93%) Other / bilingual: N ¼ 13 (7%)
184/220
197/220
187/220
N/A: not applicable.
Table 3. Ease of understanding and completion.
Measure
Very difficult
Quite difficult
10Q 10Q Parent 5Q (child rating) 5Q (Supporters’ rating) 10Q 10Q Parent 5Q (child rating) 5Q (supporters’ rating)
0 0 1 (1%) 0 0 0 0 1 (1%)
0 1 6 3 1 2 3 3
(1%) (7%) (4%) (2%) (1%) (3%) (4%)
Quite easy 15 52 26 29 10 52 27 34
(31%) (24.5%) (30%) (38%) (20%) (25%) (32%) (45%)
Very easy 34 158 53 44 38 156 55 37
(69%) (74.5%) (62%) (58%) (78%) (74%) (65%) (49%)
Total 49 (þ3 missing) 211 (þ6 missing) 86 (þ1 missing) 76 (þ11 missing) 49 (þ3 missing) 210 (þ7 missing) 85 (þ2 missing) 75 (þ12 missing)
Median rating for ease of understanding 4 4 4 4 4 4 4 3
(Very easy) (Very easy) (Very easy) (Very easy) (Very easy) (Very easy) (Very easy) (Quite easy)
Feasibility and acceptability – AHP views Among the AHPs who responded to the online survey, 84% said it was quite easy or very easy to administer the measure in their day-to-day work. Qualitative feedback from AHPs described their experiences of using the measure. Handing out the measure. The majority of AHPs reported that it was very easy to hand out the measure and explained it was ‘easy to use in clinic setting and easy to explain to clients’ (AHP 86, SLT). Some commented that having just attended a long appointment, they felt some families could be reluctant to spend any extra time completing the measure. By the end of an assessment most parents just want to get away with restless small children. (AHP 13, SLT) 7 Downloaded from chc.sagepub.com at University of Otago Library on September 28, 2015
8
Journal of Child Health Care
Table 4. Cronbach a analyses of 5Q, 10Q and 10Q Parent. Items removed for Cronbach asensitivity analyses
5Q
10Q
10Q (Parent)
No items removed Item 1 removed Item 2 removed Item 3 removed Item 4 removed Item 5 removed Item 6 removed Item 7 removed Item 8 removed Item 9 removed Item 10 removed
.746 .659 .750 .684 .701 .715 – – – – –
.926 .921 .927 .916 .917 .913 .918 .919 .915 .917 .920
.963 .961 .959 .959 .962 .957 .958 .958 .959 .957 .961
Participants’ responses. Many AHPs reported that both children and parents were happy to give feedback. Children appeared to be pleased that they were being given the opportunity to have their say. (AHP 85, OT) Parents appreciated the opportunity to feedback on their experience. (AHP 48, Physio)
Ease of understanding and completion. Many AHPs felt that children and parents found the Visual CARE Measure quick and easy to understand and complete, for example, ‘parents and children reported that it was easy to use’ (AHP 75, OT). Most AHPs felt the measure was aimed at just the right level with not too much language used. Participants with ASN. A few AHPs reflected that the measure may not be suitable for some children. I was unable to use it as the questionnaire had not been modified for the client group that I work with – Visual Impairment. (AHP 87, OT) I don’t think it would be appropriate for the majority of our school case load which is children with complex needs. (AHP 17, Physio)
Reliability Single administration reliability estimate. Internal consistency using Cronbach’s a for the 10Q was .926. Removal of any of the individual items reduced the internal consistency except for item 2; removal of this item slightly increased the internal consistency of the measure. The Cronbach’s a for the 10Q (Parent) was .963 (see Table 4) with removal of any of the questions weakening the internal reliability. The internal consistency of the 5Q using Cronbach’s a was .746. Removal of items 1, 3, 4 or 5 reduced the internal consistency, whereas removal of item 2 slightly increased the internal consistency of the measure; the difference was very slight (Table 4). Maximum possible score. On the 5Q measure, 28% of participants scored the maximum score of 25, on the 10Q measure, 35% of participants scored the maximum score of 50 and on the 10Q Parent, 54% scored the maximum score of 50 (see Table 5). 8 Downloaded from chc.sagepub.com at University of Otago Library on September 28, 2015
Place et al.
9
Table 5. Maximum score, skew, kurtosis and N/A responses. 5Q
10Q
10Q Parent
87 (83 þ 4 N/A 53 (43 þ 10 N/A 217 (185 þ 32 N/A or missing) or missing) or missing) Mean (SD) 22.2 (2.84) 45.7 (5.1) 46.3 (5.6) Min–Max 13–25 (max 28–50 (max 25–50 (max score 25) score 50) score 50) % of Respondents with max score 28% 35% 54% Skew –1.158 –1.456 –1.595 Kurtosis .902 2.277 1.829 Cronbach’s a .746 .926 .963 % N/A responses N ¼ 3 (3%) N ¼ 6 (11%) N ¼ 19 (9%) % >2 N/A N ¼ 0 (0%) N ¼ 1 (2%) N ¼ 2 (
