ORIGINAL ARTICLE: GASTROENTEROLOGY
Development and Validation of a Pediatric IBD Knowledge Inventory Device: The IBD-KID
Derek Haaland, yAndrew S. Day, and Anthony Otley
ABSTRACT Objectives: Questionnaires exist to assess inflammatory bowel disease (IBD)–related knowledge of adults. Owing to wording and content concerns, these were believed to be inappropriate for use in pediatric patients. The aim of this study was to develop a questionnaire to assess disease-related knowledge of pediatric patients with IBD and their parents. Methods: Following a formal process of item generation and reduction, the IBD-Knowledge Inventory Device was developed and pilot tested. It was administered to 10- to 17-year-old patients with IBD, and to 1 of each of their parents. To evaluate its discriminatory validity, pediatric residents, nurses, and ward clerks completed the questionnaire. Results: A total of 99 patients (mean 42, Crohn disease 46, age 14(2) years) and 99 parents completed the IBD-Knowledge Inventory Device. Parent knowledge scores, 15(4), were higher than those of patients, 11(4), P < 0.001. Patient and parent knowledge scores were strongly correlated (r ¼ 0.62, P < 0.001). Patient knowledge score was significantly related to disease type (Crohn disease scored higher than ulcerative colitis, P ¼ 0.004) and to perceived knowledge level (P < 0.001) by regression analysis. Similarly, parent knowledge score was significantly related to sex (girls scored higher, P ¼ 0.014), postsecondary education (P < 0.001), and perceived knowledge level (P ¼ 0.002). The questionnaire scores of 23 were 19, 16, and 10, respectively, for residents, nurses, and ward clerks. Both residents and nurses scored significantly higher than ward clerks (P ¼ 0.001 for both). Conclusions: A valid IBD-related knowledge assessment questionnaire was developed for use in older children and adolescents with IBD and their parents. Key Words: children, disease knowledge, IBD-KID inflammatory bowel disease
I
n recent years, there has been a trend in health care delivery to encourage patients to take an active role in their disease management. Education is an important component of patient empowerment. Patient knowledge is associated with increased treatment compliance and/or improvement of symptoms in a variety of chronic diseases (1–4). To be able to measure the efficiency of educational activities for patients with chronic illnesses and to establish the impact of these upon outcomes such as quality of life, appropriate tools are required. Inflammatory bowel disease (IBD) is a chronic disorder of unknown etiology that involves the inflammation of 1 areas of the gastrointestinal tract. Symptoms include abdominal pain, diarrhea, fever, the passage of blood or mucus per rectum, or sign of bowel obstruction (5). Two groups of investigators have developed questionnaires for the assessment of disease-related knowledge among adult patients with IBD (6,7). Jones et al (6) developed the knowledge questionnaire, a self-administered true/false questionnaire with a possible 36 correct responses covering 3 major areas: general knowledge, which included anatomy and function of bowel, extraintestinal complications, and risk to relatives; investigation, medical treatment, and surgical treatment, and their adverse effects; and patient self-management of the disease. The instrument developed by Eaden et al (7), the CCKNOW, is a self-administered 24-item multiple-choice questionnaire, which covers general IBD understanding, medication, diet, and complications; however, these devices are not suitable for children, and there is presently no equivalent instrument for pediatric patients with IBD and their parents. The purpose of the present study was to develop and validate a questionnaire to assess IBD-related knowledge among pediatric patients with IBD ages 10 to 17 years inclusive and their parents, which could then be used to assess disease-related knowledge.
(JPGN 2014;58: 313–319)
METHODS Received April 10, 2013; accepted October 9, 2013. From the Department of Pediatrics, Division of Gastroenterology, Dalhousie University, Halifax, Nova Scotia, Canada, and the yDepartment of Paediatrics, University of Otago (Christchurch), Christchurch, New Zealand. Address correspondence and reprint requests to Dr Anthony R. Otley, Department of Pediatrics, Division of Gastroenterology, IWK Health Centre, 5850 University Avenue, Halifax, NS, Canada B3K 6R8 (e-mail: [email protected]). Supplemental digital content is available for this article. Direct URL citations appear in the printed text, and links to the digital files are provided in the HTML text of this article on the journal’s Web site (www.jpgn.org). This study was supported by Medical Research Council/Burroughs Wellcome Summer Studentship, Heinish Traineeship Award and the IWK Research Foundation. The authors report no conflicts of interest. Copyright # 2014 by European Society for Pediatric Gastroenterology, Hepatology, and Nutrition and North American Society for Pediatric Gastroenterology, Hepatology, and Nutrition DOI: 10.1097/MPG.0000000000000210
JPGN
Volume 58, Number 3, March 2014
The study was divided into 3 phases: questionnaire development, questionnaire testing, and performance assessment (Fig. 1). The study was approved by the Izaak Walton Killam (IWK) research ethics board. All of the participants received a letter explaining the study and indicating that participation was voluntary and that all data would remain confidential.
Development of Questionnaire This phase of the study comprised the following stages: item generation, item reduction, pretesting, and questionnaire formatting.
Stage 1: Item Generation Domains and items for the knowledge questionnaire were based in part on previous reports (6–12), information contained in Crohn’s and Colitis Foundation of Canada (CCFC) pamphlets, and a focus group session involving 4 pediatric patients with IBD and
313
Copyright 2014 by ESPGHAN and NASPGHAN. Unauthorized reproduction of this article is prohibited.
Haaland et al
JPGN
Volume 58, Number 3, March 2014
Phase 1 - Questionnaire development Item generation • Domains established (literature review, CCFC pamphlets, focus group) • 84 questions constructed
Item reduction, pretesting, and questionnaire formatting • Reduction from 84 to 29 questions (version 1) • Evaluation by health professionals (n = 5) ð revisions ð 29 questions (version 2) • First pilot study (n = 23) ð index of difficulty and revisions ð 23 questions (version 3) • Second pilot study (n = 22) ð minor revisions ð 23 questions (version 4)
• The IBD-KID Phase 2 - Questionnaire testing (N = 198) Validity (constructs)
Reliability
• IBD-KID scores vs surrogate knowledge markers - IBD patients and parents (n = 198)
• Test-retest reliability IBD patients and parents (n = 34) • Internal consistency IBD patients and parents (n = 198)
• IBD-KID scores of pediatric residents (n = 12) vs nurses (n = 19) vs ward clerks (n = 21)
Feasibility • Readability - Flesch Reading Ease score and Flesch-Kincaid Grade Level score
• Data quality assessmentquestions answered improperly by IBD patients and parents (n = 198)
• IBD-KID scores of IBD parents vs IBD patients (n = 198) vs control parents vs control patients (n = 56)
Phase 3 - Performance assessment
• Descriptive statistics of responses to the 23 IBD-KID questions - IBD patients and parents (n = 198)
FIGURE 1. Study overview. CCFC ¼ Crohn’s and Colitis Foundation of Canada; IBD-KID ¼ Inflammatory Bowel Disease-Knowledge Inventory Device.
8 parents of pediatric patients with IBD. Domains, somewhat similar to Jones et al (6), included general information, including function and anatomy of bowel, etiology, complications, and risk to relatives; medical investigations, surgery, and medications and their adverse effects; and nutrition and lifestyle, including psychosocial issues. Initially, 84 multiple-choice and true or false questions were constructed. Each question had a ‘‘do not know’’ choice, and participants were instructed to choose this if they had no idea about the right answer. Questions used in previous IBD knowledge questionnaires (6,7) were reviewed during this process.
Stage 2: Item Reduction, Pretesting, and Questionnaire Formatting A judgmental approach (13) was used to reduce the original 84 questions to 29 questions representing all 3 domains. These questions were the first version of the questionnaire.
314
This version of the questionnaire was evaluated by 2 gastroenterologists, 2 IBD nurses, and a graduate student in the Department of Psychology at Dalhousie University. Minor changes were made based on this feedback, yielding the second version of the questionnaire. The second version of the questionnaire was piloted on 10 patients with IBD and 13 parents at the time of regularly scheduled visits to the IWK IBD clinic. Patients and their parent(s) were interviewed after completing the questionnaire to identify areas of difficulty. Each questionnaire was scored as 1 point for each correct answer, for a possible total score of 29, and an index of difficulty (the number of correct responses divided by the total number of responses) (14) was calculated for each question. With the exception of 2 questions that tested what was considered to be essential knowledge, questions leading to an index of difficulty of >0.75 for both patients and parents were eliminated or modified because they were considered poor discriminators of knowledge. Pretesting of the questionnaire with patients and parents resulted in elimination or modification of an additional 6 knowledge questions www.jpgn.org
Copyright 2014 by ESPGHAN and NASPGHAN. Unauthorized reproduction of this article is prohibited.
JPGN
Volume 58, Number 3, March 2014
based on participant feedback to improve the overall clarity. This process resulted in 23 questions, which made up the third version of the questionnaire. Two copies (1 patient copy and 1 parent copy) of the questionnaire (version 3) were mailed, accompanied by an invitation letter, to a convenience sample of 23 families with a child aged 10 to 17 years inclusive cared for by the IWK IBD team. A question about the perceived knowledge level of the participant, rated on a 5-point Likert scale, was included with each questionnaire. An additional information sheet was attached to each parent questionnaire. This sheet inquired about other relatives with IBD, parent and patient education, and family membership in the CCFC. Eleven sets of questionnaires were returned within 2 weeks. Two minor changes were made after analysis of these questionnaires, yielding the fourth and final version of the questionnaire. The questionnaire was then called the IBDKnowledge Inventory Device (IBD-KID).
Testing of the IBD-KID Questionnaire Testing of the questionnaire comprised 3 distinct stages: validity testing, reliability testing, and feasibility testing.
Validity Testing Because no external criterion for IBD-related knowledge exists, validity of the IBD-KID was assessed using a number of constructs (Table 1). The constructs were hypotheses regarding relations between IBD-KID scores in a sample of patients with IBD and their parents and presumed surrogate markers of IBD-related knowledge (6,7,15–18) and the comparison of mean IBD-KID scores among groups predicted to have disparate levels of the IBD-related knowledge (6). The IBD-KID, along with the perceived knowledge question and parent additional information sheet, was administered by mail to all of the pediatric patients with IBD ages 10 to 17 years inclusive who were cared for at the IWK IBD clinic, and their parents (n ¼ 274, parents and children). Questionnaires obtained from the second pretesting administration (see above) were also included in this analysis because version 3 of the questionnaire was essentially identical to the IBD-KID. The second group of constructs involved comparison of IBD-KID scores among groups expected to have different levels of IBD-related knowledge. Eaden et al (7) demonstrated the discriminatory value of their knowledge questionnaire by showing different mean scores among 3 groups that were expected to have different levels of IBD-related knowledge—junior doctors, registered nurses, and ward clerks. A similar method was used to assess the discriminatory value of the IBD-KID (Table 1). The IBD-KID was sent to 18 pediatric residents, 30 randomly selected nurses, and 30 randomly selected ward clerks at the IWK. As a further means of assessing the discriminatory ability of the IBD-KID, it was administered to a control population and compared with the IBD participants. The control group comprised a convenience sample of 28 non-IBD patients ages 10 to 17 years and 28 control parents during visits to either orthopedic clinics or the emergency department at the IWK.
Reliability Testing Two aspects of reliability were tested: test-retest reliability and internal consistency. An instrument is considered reliable if its results are reproducible in a group of individuals if the entity being assessed remains unchanged (19). The questionnaire was www.jpgn.org
Development of the IBD-KID administered to a convenience sample of 25 sets of patients ages 10 to 17 years and parents during a scheduled clinic appointment. Some of these individuals had completed the IBD-KID previously, but in each case, it had been several months earlier. Participants were given a second set of questionnaires and asked to complete them a minimum of 2 and a maximum of 4 weeks following the appointment and then return them by mail. The timespan between first and second administration was believed to be long enough to decrease the possibility of memorization of previous answers, but short enough that disease-related knowledge was expected to remain unchanged. Participants who had not returned their questionnaires shortly after 2 weeks were contacted by telephone. The second method of reliability assessment was measurement of internal consistency using Cronbach a (19). This measure is a function of the number of items and the average correlation between pairs of items on a scale, and is an indicator of the extent to which different items on a scale are assessing a single entity. The data obtained from the validity testing of patients with IBD and their parents were also used for this purpose.
Feasibility Testing Feasibility assesses the ease with which a device can be administered by researchers and completed by subjects. Two aspects of feasibility were assessed: readability and data quality. Readability of the IBD-KID was calculated using the Flesch Reading Ease score and the Flesch-Kincaid Grade Level score (20). Data quality was assessed by determining the percentage of questions left blank or answered improperly (eg, >1 answer circled) (21).
Performance Assessment of the IBD-KID The performance of pediatric patients and their parents on the individual IBD-KID questions was assessed (ie, number correct versus incorrect).
Statistical Analysis All statistical analyses were performed using SPSS for Windows, version 9.0.0 (SPSS Inc, Chicago, IL) (22). Results were considered significant with P < 0.05. Associations between both patient and parent IBD-KID scores and a number of independent variables were evaluated in bivariate or multivariate analyses. The independent variables were those listed in Table 1. t Tests and Pearson correlation coefficients were used for comparisons using dichotomous and continuous independent variables, respectively. Two backward multiple linear regression analyses were performed: 1 with patient knowledge score as the dependent variable and 1 with parent knowledge score as the dependent variable. Independent variables were the same as those used in univariate analyses. In each case, means were compared using a 1-way analysis of variance Tukey test. In terms of the analysis of the reliability phase, intraclass correlation coefficients were calculated for parents and patients separately, and for the entire group, to calculate Cronbach a for the entire group and for patients and parents separately.
RESULTS Testing of the IBD-KID Questionnaire Validity of the IBD-KID Questionnaire For the validity assessment, a total of 137 pairs of questionnaires were sent to patients with IBD and their parents,
315
Copyright 2014 by ESPGHAN and NASPGHAN. Unauthorized reproduction of this article is prohibited.
Haaland et al
JPGN
Volume 58, Number 3, March 2014
TABLE 1. A priori constructs to assess validity and discriminatory ability of the IBD-KID
Construct
Resulty
Statistic
Association of IBD-KID scores of patients with IBD with specific variables Scores should correlate positively with perceived Pearson correlation coefficient knowledge level Scores should correlate positively with parent scores Pearson correlation coefficient
Scores should correlate positively with disease duration Scores should correlate positively with patient age Patients who have completed grade 8 should score higher
Pearson correlation coefficient Pearson correlation coefficient t test
Patients whose parent has postsecondary education should score higher Patients with CCFC membership should score higher
t test
Patients who have a close relative with IBD should score higher
t test
Scores should not differ between male and female patients
t test
Scores should not differ between CD and UC patients
t test
Association of IBD parent IBD-KID scores with specific variables Scores should correlate positively with perceived knowledge level Scores should correlate positively with patient scores
t test
Pearson correlation coefficient Pearson correlation coefficient
Scores should correlate positively with disease duration Parents with postsecondary education should score higher
Pearson correlation coefficient t test
Parents with CCFC membership should score higher
t test
Parents who have a close relative with IBD should score higher IBD-KID mean scores and level of professional training IBD-KID scores should be as follows: pediatric residents > nurses > ward clerks
t test
IBD-KID mean scores among IBD and control populationsz IBD-KID scores should be as follows: IBD parents > Patients with IBD >control parents > control patients
Pearson coefficient ¼ 0.43 P < 0.001 Pearson coefficient ¼ 0.49 P < 0.001 P ¼ NS P ¼ NS >G8: 11.60 0.47; G8 10.85 0.60 P ¼ NS Yes: 11.52 0.45; no: 10.81 0.66 P ¼ NS Yes: 12.61 0.84; no: 10.89 0.40 P ¼ 0.025 Yes: 11.44 0.50; no: 11.19 0.53 P ¼ NS Male 11.1 3.7 (n ¼ 42); female 11.8 4.2 (n ¼ 36), P ¼ NS CD: 12.4 4.0 (n ¼ 46); UC 10.1 3.4 (n ¼ 32), P ¼ 0.01 Pearson coefficient ¼ 0.40 P < 0.001 Pearson coefficient ¼ 0.49 P < 0.001 P ¼ NS Yes: 15.45 0.44; no: 12.69 0.63 P < 0.001 Yes: 15.08 0.86; no: 14.39 0.42 P ¼ NS Yes: 14.61 0.55; no: 14.52 0.53 P ¼ NS
Tukey
Residents ¼ 19.08 0.84 (n ¼ 12) Nurses ¼ 15.79 0.64 (n ¼ 19) Ward clerks ¼ 9.24 1.07 (n ¼ 21) P < 0.001 except: P ¼ 0.03 for nurses vs ward clerks
Tukey
IBD parents ¼ 14.56 0.38 (n ¼ 99) Patients with IBD ¼ 11.29 0.37 (n ¼ 99) Control parents ¼ 7.43 0.71 (n ¼ 28) Control patients ¼ 4.82 0.63 (n ¼ 28) P < 0.001 except: P ¼ 0.02 control parents vs control patients
CCFC ¼ Crohn’s and Colitis Foundation of Canada; CD ¼ Crohn disease; IBD-KID ¼ Inflammatory Bowel Disease-Knowledge Inventory Device; NS ¼ not significant; UC ¼ ulcerative colitis. IBD patient and parent variables were also entered into separate backward multiple linear regression models (see text). y See text for results of multiple linear regression models. z Control comparisons were not adjusted for demographic variables (eg, age, education).
of which 99 pairs were returned, yielding a response rate of 72%. Table 2 describes the demographics and disease characteristics of the patient population. Of the 99 parents who responded, 87 (88%) were female and 67 (68%) had completed some form of postsecondary education. Wherein specific differences were predicted, bivariate analyses of patient scores and parent scores are presented in Table 1. Patients with Crohn disease (CD) scored higher than patients with ulcerative colitis (UC) (11.39 0.49 vs 10.35 0.54, P ¼ 0.035). This did not hold true for parents of patients with CD versus parents of patients with UC (P ¼ NS). IBD-KID scores
316
did not differ between male and female patients; however, female parents scored higher than male parents (14.93 0.38 vs 11.83 1.25; P ¼ 0.007). In multivariate analysis of patient IBD-KID scores, perceived knowledge and parent total score were significant (P < 0.001) and disease type bordered on significance (P ¼ 0.052) with patients with CD scoring higher. The model including these 3 variables yielded an adjusted R2 0.357 (P < 0.001). Significant variables in multivariate analysis of parent IBDKID scores were parent postsecondary education (P < 0.001), patient total (P < 0.001), perceived knowledge level (P ¼ 0.001), www.jpgn.org
Copyright 2014 by ESPGHAN and NASPGHAN. Unauthorized reproduction of this article is prohibited.
JPGN
Volume 58, Number 3, March 2014
Development of the IBD-KID
TABLE 2. Demographics of patients with IBD N ¼ 99 Sex Age (mean SD), y School grade Disease type Disease duration (mean SD), y CCFC membership Relative with IBD (%)
Male ¼ 52; female ¼ 47 14.6 0.2 Grade 8 ¼ 41 CD ¼ 59; UC ¼ 40 3.2 0.2 Yes ¼ 23; no ¼ 76 41 (41)
CCFC ¼ Crohn’s and Colitis Foundation of Canada; CD ¼ Crohn disease; IBD ¼ inflammatory bowel disease; SD ¼ standard deviation; UC ¼ ulcerative colitis. Relative with IBD indicated parent with first-degree relative having IBD.
and parent sex (women scored higher; P ¼ 0.021). The adjusted R2 for the model was 0.428 (P < 0.001). The response rates of the pediatric residents, nurses, and ward clerks were 67%, 63%, and 70%, respectively. A comparison of scores obtained by these 3 groups, as well as by parents of patients with IBD, patients with IBD, control parents, and control patients, is presented in Table 1.
Reliability of the IBD-KID Questionnaire Seventeen patients and parents completed 2 sets of questionnaires within 2 to 4 weeks. The intraclass correlation was 0.84 (95% confidence interval 0.70–0.92). The internal consistency was measured using Cronbach a. The Cronbach a was 0.75 (95% confidence interval 0.69–0.80).
Feasibility of the IBD-KID Questionnaire The Flesch Reading Ease score was 69.0, and the FleschKincaid Grade Level score was 6.3. Of the total questions answered, only 0.9% (42/4554) were completed improperly (eg, left blank, >1 answer circled). The values for patients and parents were 1.1% (26/2277) and 0.7% (16/2277), respectively.
Performance of pediatric patients with IBD and their parents on selected IBD-KID questions is presented in Table 3. A copy of the IBD-KID assessment is provided online, http://links.lww.com/ MPG/A275.
DISCUSSION This is the first IBD-specific knowledge questionnaire designed for and validated in a pediatric IBD population. This questionnaire was developed solely in a pediatric setting, and was established to provide a valid and reliable assessment of knowledge in these children and their parents. The questionnaire was validated using a number of methods. First, scores were predicted to correlate with several variables a priori. In univariate and regression analyses, as expected, patient and parent scores correlated with perceived knowledge level and with respective parent and patient scores. In univariate but not regression analyses, parent postsecondary education correlated with parent scores (17,23), but not patient scores. The association in univariate analysis was similar to the findings of Jones et al (6). In univariate but not regression analyses, CCFC membership correlated with patient, but not parent, score. Previous adult studies (6,7) have found that patients with membership in the Crohn’s and Colitis UK scored higher on knowledge questionnaires. This suggests that these individuals are more interested in learning about their disease and/or have greater exposure to educational material through their membership. In univariate analysis, the IBD-KID score was higher among patients with CD versus UC, and this bordered on significance in regression analysis (P ¼ 0.052). This finding was not predicted, but has been noted previously (6,23). This may relate to the fact that CD can run a more severe course than UC and that patients with CD report worse health-related quality of life than those with UC (24). Furthermore, CD has a greater likelihood of systemic manifestations, more extraintestinal manifestations, and higher rate of growth retardation (25) and is associated with more pain (26). This relation between level of knowledge and disease burden has been demonstrated in other disease entities. Among patients with type 1 diabetes mellitus, it was found that a knowledge deficit was associated with a lack of interest in disease information when feeling well (27).
TABLE 3. Performance of pediatric patients with IBD and their parents on selected IBD-KID questions No. correct responses, N ¼ 99 Concept No risk of IBD transmission The etiology of IBD is unknown IBD is familial Most patients with IBD have normal life expectancy Emotional stress and IBD flare-up Correct order of digestive tract IBD can affect other organs Risk of colon cancer in IBD No foods proven to cause IBD flare-ups Osteoporosis is related to IBD, steroids, and poor nutrition Herbal remedies can have adverse effects and interact with drugs Role of enteral nutrition (treatment and prevention of CD/growth) Role of surgery in CD and UC Role of salicylates and azathioprine in maintenance therapy
Patient
Parent
90 78 71 70 69 62 47 37 36 29 24 20 16 9
96 89 74 62 79 78 70 49 50 49 32 41 32 23
CD ¼ Crohn disease; IBD-KID ¼ Inflammatory Bowel Disease-Knowledge Inventory Device; UC ¼ ulcerative colitis.
www.jpgn.org
317
Copyright 2014 by ESPGHAN and NASPGHAN. Unauthorized reproduction of this article is prohibited.
Haaland et al Surprisingly, there was no association between knowledge score and days since diagnosis—as some previous adult asthma and diabetes literature has found—the longer the duration of disease, the better the disease knowledge (17,18). This finding did, however, confirm previous results among adult patients (7,28). It is possible that core information is obtained shortly after diagnosis, with minimal new information being attained subsequently or forgotten over time. There was no association between patient IBD-KID score and age, which was also the case with medication knowledge in youth with IBD (29) and similar to results in adult patients with rheumatoid arthritis (28); however, in a study of pediatric and young adult patients with asthma, age was the most significant predictor of asthma knowledge (15). There was also no association with school grade, which may be explained by the lack of association between knowledge and age. Another unpredicted finding in the present study was that mothers scored higher than fathers in univariate and regression analyses. Mothers were also far more likely than fathers to be the parent who chose to complete the questionnaire (87 vs 12 respondents). Similar results were found among the parents of patients with cystic fibrosis (16). These findings may be explained in part by literature that suggests that the impact of a chronically ill child is experienced more greatly by the mother than by the father and that fathers, more often than mothers, describe themselves as outside observers in terms of the management of their child’s illness (30). Further validation was achieved through demonstration that, as predicted, mean scores of pediatric residents, nurses, and ward clerks differed significantly. A comparable method of validation was used previously (7). Similarly, as hypothesized, significantly different scores were obtained from parents of patients with IBD, patients with IBD, control parents, and control patients. This ability to distinguish among groups based on hypothesized knowledge level is the strength of this device. The questionnaire also demonstrated excellent test-retest reliability (21). The Cronbach a of the IBD-KID (0.75) was lower than those of previous IBD knowledge questionnaires (0.84 obtained by Jones et al (6); 0.95 obtained by Eaden et al (7)). A value of >0.70 is recommended for comparison among groups of patients, and >0.90 is suggested for analyzing individual patient scores (31). Cronbach a is a useful measure, but it is not fully relevant when applied to a knowledge questionnaire because disease-related knowledge is not truly a single entity, but rather is multifaceted. For instance, it is quite reasonable that a given individual may know a lot about medication adverse effects, but be relatively uninformed about disease complications. The IBD-KID scored extremely well in terms of feasibility for both patients and parents. The Flesch Reading Ease score for the questionnaire is 69, which is considered to be easier to understand than ‘‘plain English.’’ The Flesch-Kincaid Grade Level score is grade 6. Because the questionnaire was targeted toward pediatric patients with IBD ages 10 to 17 and their parents, the ideal FleschKincaid Grade Level score for the questionnaire would thus be in the range of 4 to 5; however, IBD-related terms including ulcerative colitis, osteoporosis, colonoscopy, and drug names such as sulfasalazine and azathioprine were purposefully included in the questionnaire because it was believed that an understanding of such terms is an essential component of overall knowledge of IBD. Because the average number of syllables per word factors heavily into the Flesch Reading Ease score, the inclusion of these terms adversely affected this score. It is reassuring that patient score did not correlate with age or grade, suggesting that reading ability did not hinder performance among younger patients. This study revealed several important knowledge gaps among the Maritime Canadian pediatric patient population with
318
JPGN
Volume 58, Number 3, March 2014
IBD and their parents. For instance, only approximately half of the parents and approximately one-third of patients were aware that IBD, corticosteroid use, and poor nutrition could result in osteoporosis. In their assessment of knowledge in adult patients, Jones et al (6) found that only 45% of patients knew that steroids could cause osteoporosis. In the present study, just 35% of patients and 41% of parents were aware that herbal medications have the potential to interact with conventional drugs; however, 70% of parents but only 47% of patients were aware that IBD may affect organs other than the bowels. Eaden et al (7) found that 53% of adult patients were aware of this fact. Sixty-three percent of patients and 47% of adults either believed that removing certain foods from the diet would prevent flare-ups of IBD or were unsure. A possible limitation of this instrument may be the need to adapt or change the questionnaire over time. We also did not measure disease severity; it would be beneficial to include this variable in future studies. In a study of pediatric and young adult patients with asthma, it was found that more severe disease was positively related to knowledge (15). Lastly, because every parent– child pair completed the questionnaire, we cannot rule out the possibility that the child did not complete the assessment of their own accord, but rather was encouraged by the parent; however, parents and patients were instructed to complete the questionnaire separately and to check the ‘‘do not know’’ category when unsure of an answer, to reduce pressure. Using a pediatric-specific questionnaire should reduce bias by including questions that are relevant to the population, and by formatting/writing questions that maximize readability and ease of use for a young audience. Thus, although some bias is possible given the design of the present study, the methodology of questionnaire development also reduced other sources of bias. In conclusion, the IBD-KID is a valid and useful tool for the evaluation of IBD-related knowledge among pediatric patients with IBD and their parents. High levels of disease-related knowledge have been associated with improved disease outcomes (32,33). A recent study among adult patients found that greater IBD knowledge was associated with the use of more adaptive coping strategies (34). Furthermore, other adult reports suggest that providing IBD education reduces medication nonadherence (35,36) and that treatment adherence is positively associated with patient awareness of their disease and medications (37). The IBD-KID may also encourage self-management and aid in the transition of patients from pediatric to adult care by identifying and promoting awareness of areas of knowledge paucity. Future work with the pediatric population could entail carrying out an experimental study to look at the cause-and-effect relation between increasing disease knowledge and disease outcomes such as medication adherence and self-management. This study identified many important knowledge deficits in the study population. It has been reported that there exists a lack of medical knowledge in IBD transitioned and adolescent patients (38–40) and that many adolescent patients with IBD are unaware of the adverse effects of their IBD medications (29). Acknowledgments: The authors thank the hospital staff, patients, and their parents for their assistance with this study.
REFERENCES 1. Hughes DM, McLeod M, Garner B, et al. Controlled trial of a home and ambulatory program for asthmatic children. Pediatrics 1991;87: 54–61. 2. Couper JJ, Taylor J, Fotheringham MJ, et al. Failure to maintain the benefits of home-based intervention in adolescents with poorly controlled type 1 diabetes. Diabetes Care 1999;22:1933–7.
www.jpgn.org
Copyright 2014 by ESPGHAN and NASPGHAN. Unauthorized reproduction of this article is prohibited.
JPGN
Volume 58, Number 3, March 2014
3. Lorig K, FP, Regan C, Ung E, et al. A comparison of lay-taught and professional-taught arthritis self-management courses. J Rheumatol 1986;13:763–7. 4. Brown S. Effects of educational interventions in diabetes care: a metaanalysis of findings. Nurs Res 1988;37:223–30. 5. Kappelman MD, Rifas-Shiman SL, Kleinman K, et al. The prevalence and geographic distribution of Crohn’s disease and ulcerative colitis in the United States. Clin Gastroenterol Hepatol 2007;5:1424–9. 6. Jones SC, Gallacher B, Lobo AJ, et al. A patient knowledge questionnaire in inflammatory bowel disease. J Clin Gastroenterol 1993;17:21–4. 7. Eaden JA, Abrams K, Mayberry JF. The Crohn’s and Colitis Knowledge Score: a test for measuring patient knowledge in inflammatory bowel disease. Am J Gastroenterol 1999;94:3560–6. 8. Mansfield JC, Tanner AR, Bramble MG. Information for patients about inflammatory bowel disease. J R Coll Phys London 1997;31:184–7. 9. Eaden JA, Ward B, Smith H, et al. Are we telling patients enough? A pilot study to assess patient information needs in a gastroenterology outpatient department. Eur J Gastroenterol Hepatol 1998;10:63–7. 10. Hilsden RJ, Dunn C, Patten S, et al. Information needs and seeking behaviors of patients with inflammatory bowel disease. Gastroenterology 1998; Abstract G4078:A995. 11. Scholmerich J, Sedlak P, Hoppe-Seyler P, et al. The information needs and fears of patients with inflammatory bowel disease. Hepatogastroenterology 1987;34:182–5. 12. Moser G, Tillinger W, Sachs G, et al. Disease-related worries and concerns: a study on out-patients with inflammatory bowel disease. Eur J Gastroenterol Hepatol 1995;7:853–8. 13. Otley AR. Validation of the IMPACT Questionnaire for Measuring Health-Related Quality of Life in Paediatric Inflammatory Bowel Disease: Sensibility, Validity and Reliability Testing in a Paediatric Cohort [Dissertation] Toronto: University of Toronto; 2004. 14. Hubbard JPCW. Multiple Choice Examinations in Medicine. London: Henry Kimpton; 1961. 15. Barton C, Abramson M, Aroni R, et al. What determines knowledge of asthma among young people and their families? J Asthma 2002;39: 701–9. 16. Nolan T, Desmond K, Herlich R, et al. Knowledge of cystic fibrosis in patients and their parents. Pediatrics 1986;77:229–35. 17. Emtner M, Hedin A, Andersson M, et al. Impact of patient characteristics, education and knowledge on emergency room visits in patients with asthma and COPD: a descriptive and correlative study. BMC Pulm Med 2009;9:43. 18. Firestone DN, Jimenez-Briceno L, Reimann JO, et al. Predictors of diabetes-specific knowledge and treatment satisfaction among Costa Ricans. Diabetes Educ 2004;30:281–92. 19. Streiner DL, Norman GR. Health Measurement Scales: A Practical Guide to Their Development and Use. 2nd ed. New York: Oxford University Press; 1995. 20. Flesch RF. A new readability yardstick. J Appl Psychol 1948;32: 221–33. 21. McHorney CA, Ware JE, Lu JFR, et al. The MOS 36-Item Short-Form Health Survey (SF-36): III. Tests of Data quality, scaling assumptions, and reliability across diverse patient groups. Med Care 1994;32:40–66. 22. SPSS (for Windows) [computer program]. Version 10.0.5. Chicago, IL: SPSS Inc; 1999.
www.jpgn.org
Development of the IBD-KID
23. Subasinghe D, Wijekoon NS, Nawarathne NM, et al. Disease-related knowledge in inflammatory bowel disease: experience of a tertiary care centre in a developing country in South Asia. Singapore Med J 2014;51:484–9. 24. Larsson K, Loof L, Ronnblom A, et al. Quality of life for patients with exacerbation in inflammatory bowel disease and how they cope with disease activity. J Psychosom Res 2008;64:139–48. 25. Griffiths AM. Inflammatory bowel disease. Adolesc Med State Art Rev 1995;6:351–68. 26. Lix LM, Graff LA, Walker JR, et al. Longitudinal study of quality of life and psychological functioning for active, fluctuating, and inactive disease patterns in inflammatory bowel disease. Inflamm Bowel Dis 2008;14:1575–84. 27. Rankin D, Heller S, Lawton J. Understanding information and education gaps among people with type 1 diabetes: a qualitative investigation. Patient Educ Couns 2011;83:87–91. 28. Hennell SL, Brownsell C, Dawson JK. Development, validation and use of a patient knowledge questionnaire (PKQ) for patients with early rheumatoid arthritis. Rheumatology 2004;43:467–71. 29. Fishman LN, Houtman D, van Groningen J, et al. Medication knowledge: an initial step in self-management for youth with inflammatory bowel disease. J Pediatr Gastroenterol Nutr 2011;53:641–5. 30. Knafl K, Zoeller L. Childhood chronic illness: a comparison of mothers’ and fathers’ experiences. J Fam Nurs 2000;6:287–302. 31. Portney LG, Watkins MP. Statistical measures of reliability. In: Foundation of Clinical Research: Applications to Practice. Norwich, CT: Appleton and Lange; 1993. 32. Hampson SE, Skinner TC, Hart J, et al. Effects of educational and psychosocial interventions for adolescents with diabetes mellitus: a systematic review. Health Technol Assess 2001;5:1–79. 33. Loveman E, Cave C, Green C, et al. The clinical and cost-effectiveness of patient education models for diabetes: a systematic review and economic evaluation. Health Technol Assess 2003;7:1–190iii. 34. Moradkhani A, Kerwin L, Dudley-Brown S, et al. Disease-specific knowledge, coping, and adherence in patients with inflammatory bowel disease. Dig Dis Sci 2011;56:2972–7. 35. Waters BM, Jensen L, Fedorak RN. Effects of formal education for patients with inflammatory bowel disease: a randomized controlled trial. Can J Gastroenterol 2005;19:235–44. 36. Hawthorne AB, Rubin G, Ghosh S. Review article: medication nonadherence in ulcerative colitis: strategies to improve adherence with mesalazine and other maintenance therapies. Aliment Pharmacol Ther 2008;27:1157–66. ˇ erveny´ P, Bortlı´k M, Kubeˇna A, et al. Nonadherence in inflammatory 37. C bowel disease: results of factor analysis. Inflamm Bowel Dis 2007;13: 1244–9. 38. Hait EJ, Barendse ReM, Arnold JH, et al. Transition of adolescents with inflammatory bowel disease from pediatric to adult care: a survey of adult gastroenterologists. J Pediatr Gastroenterol Nutr 2009;48:61–5. 39. Fishman LN, Barendse ReM, Hait E, et al. Self-management of older adolescents with inflammatory bowel disease: a pilot study of behavior and knowledge as prelude to transition. Clin Pediatr 2010;49:1129–33. 40. Benchimol EI, Walters TD, Kaufman M, et al. Assessment of knowledge in adolescents with inflammatory bowel disease using a novel transition tool. Inflamm Bowel Dis 2011;17:1131–7.
319
Copyright 2014 by ESPGHAN and NASPGHAN. Unauthorized reproduction of this article is prohibited.
Development and Validation of a Pediatric IBD Knowledge Inventory Device: The IBD-KID
Derek Haaland, yAndrew S. Day, and Anthony Otley
ABSTRACT Objectives: Questionnaires exist to assess inflammatory bowel disease (IBD)–related knowledge of adults. Owing to wording and content concerns, these were believed to be inappropriate for use in pediatric patients. The aim of this study was to develop a questionnaire to assess disease-related knowledge of pediatric patients with IBD and their parents. Methods: Following a formal process of item generation and reduction, the IBD-Knowledge Inventory Device was developed and pilot tested. It was administered to 10- to 17-year-old patients with IBD, and to 1 of each of their parents. To evaluate its discriminatory validity, pediatric residents, nurses, and ward clerks completed the questionnaire. Results: A total of 99 patients (mean 42, Crohn disease 46, age 14(2) years) and 99 parents completed the IBD-Knowledge Inventory Device. Parent knowledge scores, 15(4), were higher than those of patients, 11(4), P < 0.001. Patient and parent knowledge scores were strongly correlated (r ¼ 0.62, P < 0.001). Patient knowledge score was significantly related to disease type (Crohn disease scored higher than ulcerative colitis, P ¼ 0.004) and to perceived knowledge level (P < 0.001) by regression analysis. Similarly, parent knowledge score was significantly related to sex (girls scored higher, P ¼ 0.014), postsecondary education (P < 0.001), and perceived knowledge level (P ¼ 0.002). The questionnaire scores of 23 were 19, 16, and 10, respectively, for residents, nurses, and ward clerks. Both residents and nurses scored significantly higher than ward clerks (P ¼ 0.001 for both). Conclusions: A valid IBD-related knowledge assessment questionnaire was developed for use in older children and adolescents with IBD and their parents. Key Words: children, disease knowledge, IBD-KID inflammatory bowel disease
I
n recent years, there has been a trend in health care delivery to encourage patients to take an active role in their disease management. Education is an important component of patient empowerment. Patient knowledge is associated with increased treatment compliance and/or improvement of symptoms in a variety of chronic diseases (1–4). To be able to measure the efficiency of educational activities for patients with chronic illnesses and to establish the impact of these upon outcomes such as quality of life, appropriate tools are required. Inflammatory bowel disease (IBD) is a chronic disorder of unknown etiology that involves the inflammation of 1 areas of the gastrointestinal tract. Symptoms include abdominal pain, diarrhea, fever, the passage of blood or mucus per rectum, or sign of bowel obstruction (5). Two groups of investigators have developed questionnaires for the assessment of disease-related knowledge among adult patients with IBD (6,7). Jones et al (6) developed the knowledge questionnaire, a self-administered true/false questionnaire with a possible 36 correct responses covering 3 major areas: general knowledge, which included anatomy and function of bowel, extraintestinal complications, and risk to relatives; investigation, medical treatment, and surgical treatment, and their adverse effects; and patient self-management of the disease. The instrument developed by Eaden et al (7), the CCKNOW, is a self-administered 24-item multiple-choice questionnaire, which covers general IBD understanding, medication, diet, and complications; however, these devices are not suitable for children, and there is presently no equivalent instrument for pediatric patients with IBD and their parents. The purpose of the present study was to develop and validate a questionnaire to assess IBD-related knowledge among pediatric patients with IBD ages 10 to 17 years inclusive and their parents, which could then be used to assess disease-related knowledge.
(JPGN 2014;58: 313–319)
METHODS Received April 10, 2013; accepted October 9, 2013. From the Department of Pediatrics, Division of Gastroenterology, Dalhousie University, Halifax, Nova Scotia, Canada, and the yDepartment of Paediatrics, University of Otago (Christchurch), Christchurch, New Zealand. Address correspondence and reprint requests to Dr Anthony R. Otley, Department of Pediatrics, Division of Gastroenterology, IWK Health Centre, 5850 University Avenue, Halifax, NS, Canada B3K 6R8 (e-mail: [email protected]). Supplemental digital content is available for this article. Direct URL citations appear in the printed text, and links to the digital files are provided in the HTML text of this article on the journal’s Web site (www.jpgn.org). This study was supported by Medical Research Council/Burroughs Wellcome Summer Studentship, Heinish Traineeship Award and the IWK Research Foundation. The authors report no conflicts of interest. Copyright # 2014 by European Society for Pediatric Gastroenterology, Hepatology, and Nutrition and North American Society for Pediatric Gastroenterology, Hepatology, and Nutrition DOI: 10.1097/MPG.0000000000000210
JPGN
Volume 58, Number 3, March 2014
The study was divided into 3 phases: questionnaire development, questionnaire testing, and performance assessment (Fig. 1). The study was approved by the Izaak Walton Killam (IWK) research ethics board. All of the participants received a letter explaining the study and indicating that participation was voluntary and that all data would remain confidential.
Development of Questionnaire This phase of the study comprised the following stages: item generation, item reduction, pretesting, and questionnaire formatting.
Stage 1: Item Generation Domains and items for the knowledge questionnaire were based in part on previous reports (6–12), information contained in Crohn’s and Colitis Foundation of Canada (CCFC) pamphlets, and a focus group session involving 4 pediatric patients with IBD and
313
Copyright 2014 by ESPGHAN and NASPGHAN. Unauthorized reproduction of this article is prohibited.
Haaland et al
JPGN
Volume 58, Number 3, March 2014
Phase 1 - Questionnaire development Item generation • Domains established (literature review, CCFC pamphlets, focus group) • 84 questions constructed
Item reduction, pretesting, and questionnaire formatting • Reduction from 84 to 29 questions (version 1) • Evaluation by health professionals (n = 5) ð revisions ð 29 questions (version 2) • First pilot study (n = 23) ð index of difficulty and revisions ð 23 questions (version 3) • Second pilot study (n = 22) ð minor revisions ð 23 questions (version 4)
• The IBD-KID Phase 2 - Questionnaire testing (N = 198) Validity (constructs)
Reliability
• IBD-KID scores vs surrogate knowledge markers - IBD patients and parents (n = 198)
• Test-retest reliability IBD patients and parents (n = 34) • Internal consistency IBD patients and parents (n = 198)
• IBD-KID scores of pediatric residents (n = 12) vs nurses (n = 19) vs ward clerks (n = 21)
Feasibility • Readability - Flesch Reading Ease score and Flesch-Kincaid Grade Level score
• Data quality assessmentquestions answered improperly by IBD patients and parents (n = 198)
• IBD-KID scores of IBD parents vs IBD patients (n = 198) vs control parents vs control patients (n = 56)
Phase 3 - Performance assessment
• Descriptive statistics of responses to the 23 IBD-KID questions - IBD patients and parents (n = 198)
FIGURE 1. Study overview. CCFC ¼ Crohn’s and Colitis Foundation of Canada; IBD-KID ¼ Inflammatory Bowel Disease-Knowledge Inventory Device.
8 parents of pediatric patients with IBD. Domains, somewhat similar to Jones et al (6), included general information, including function and anatomy of bowel, etiology, complications, and risk to relatives; medical investigations, surgery, and medications and their adverse effects; and nutrition and lifestyle, including psychosocial issues. Initially, 84 multiple-choice and true or false questions were constructed. Each question had a ‘‘do not know’’ choice, and participants were instructed to choose this if they had no idea about the right answer. Questions used in previous IBD knowledge questionnaires (6,7) were reviewed during this process.
Stage 2: Item Reduction, Pretesting, and Questionnaire Formatting A judgmental approach (13) was used to reduce the original 84 questions to 29 questions representing all 3 domains. These questions were the first version of the questionnaire.
314
This version of the questionnaire was evaluated by 2 gastroenterologists, 2 IBD nurses, and a graduate student in the Department of Psychology at Dalhousie University. Minor changes were made based on this feedback, yielding the second version of the questionnaire. The second version of the questionnaire was piloted on 10 patients with IBD and 13 parents at the time of regularly scheduled visits to the IWK IBD clinic. Patients and their parent(s) were interviewed after completing the questionnaire to identify areas of difficulty. Each questionnaire was scored as 1 point for each correct answer, for a possible total score of 29, and an index of difficulty (the number of correct responses divided by the total number of responses) (14) was calculated for each question. With the exception of 2 questions that tested what was considered to be essential knowledge, questions leading to an index of difficulty of >0.75 for both patients and parents were eliminated or modified because they were considered poor discriminators of knowledge. Pretesting of the questionnaire with patients and parents resulted in elimination or modification of an additional 6 knowledge questions www.jpgn.org
Copyright 2014 by ESPGHAN and NASPGHAN. Unauthorized reproduction of this article is prohibited.
JPGN
Volume 58, Number 3, March 2014
based on participant feedback to improve the overall clarity. This process resulted in 23 questions, which made up the third version of the questionnaire. Two copies (1 patient copy and 1 parent copy) of the questionnaire (version 3) were mailed, accompanied by an invitation letter, to a convenience sample of 23 families with a child aged 10 to 17 years inclusive cared for by the IWK IBD team. A question about the perceived knowledge level of the participant, rated on a 5-point Likert scale, was included with each questionnaire. An additional information sheet was attached to each parent questionnaire. This sheet inquired about other relatives with IBD, parent and patient education, and family membership in the CCFC. Eleven sets of questionnaires were returned within 2 weeks. Two minor changes were made after analysis of these questionnaires, yielding the fourth and final version of the questionnaire. The questionnaire was then called the IBDKnowledge Inventory Device (IBD-KID).
Testing of the IBD-KID Questionnaire Testing of the questionnaire comprised 3 distinct stages: validity testing, reliability testing, and feasibility testing.
Validity Testing Because no external criterion for IBD-related knowledge exists, validity of the IBD-KID was assessed using a number of constructs (Table 1). The constructs were hypotheses regarding relations between IBD-KID scores in a sample of patients with IBD and their parents and presumed surrogate markers of IBD-related knowledge (6,7,15–18) and the comparison of mean IBD-KID scores among groups predicted to have disparate levels of the IBD-related knowledge (6). The IBD-KID, along with the perceived knowledge question and parent additional information sheet, was administered by mail to all of the pediatric patients with IBD ages 10 to 17 years inclusive who were cared for at the IWK IBD clinic, and their parents (n ¼ 274, parents and children). Questionnaires obtained from the second pretesting administration (see above) were also included in this analysis because version 3 of the questionnaire was essentially identical to the IBD-KID. The second group of constructs involved comparison of IBD-KID scores among groups expected to have different levels of IBD-related knowledge. Eaden et al (7) demonstrated the discriminatory value of their knowledge questionnaire by showing different mean scores among 3 groups that were expected to have different levels of IBD-related knowledge—junior doctors, registered nurses, and ward clerks. A similar method was used to assess the discriminatory value of the IBD-KID (Table 1). The IBD-KID was sent to 18 pediatric residents, 30 randomly selected nurses, and 30 randomly selected ward clerks at the IWK. As a further means of assessing the discriminatory ability of the IBD-KID, it was administered to a control population and compared with the IBD participants. The control group comprised a convenience sample of 28 non-IBD patients ages 10 to 17 years and 28 control parents during visits to either orthopedic clinics or the emergency department at the IWK.
Reliability Testing Two aspects of reliability were tested: test-retest reliability and internal consistency. An instrument is considered reliable if its results are reproducible in a group of individuals if the entity being assessed remains unchanged (19). The questionnaire was www.jpgn.org
Development of the IBD-KID administered to a convenience sample of 25 sets of patients ages 10 to 17 years and parents during a scheduled clinic appointment. Some of these individuals had completed the IBD-KID previously, but in each case, it had been several months earlier. Participants were given a second set of questionnaires and asked to complete them a minimum of 2 and a maximum of 4 weeks following the appointment and then return them by mail. The timespan between first and second administration was believed to be long enough to decrease the possibility of memorization of previous answers, but short enough that disease-related knowledge was expected to remain unchanged. Participants who had not returned their questionnaires shortly after 2 weeks were contacted by telephone. The second method of reliability assessment was measurement of internal consistency using Cronbach a (19). This measure is a function of the number of items and the average correlation between pairs of items on a scale, and is an indicator of the extent to which different items on a scale are assessing a single entity. The data obtained from the validity testing of patients with IBD and their parents were also used for this purpose.
Feasibility Testing Feasibility assesses the ease with which a device can be administered by researchers and completed by subjects. Two aspects of feasibility were assessed: readability and data quality. Readability of the IBD-KID was calculated using the Flesch Reading Ease score and the Flesch-Kincaid Grade Level score (20). Data quality was assessed by determining the percentage of questions left blank or answered improperly (eg, >1 answer circled) (21).
Performance Assessment of the IBD-KID The performance of pediatric patients and their parents on the individual IBD-KID questions was assessed (ie, number correct versus incorrect).
Statistical Analysis All statistical analyses were performed using SPSS for Windows, version 9.0.0 (SPSS Inc, Chicago, IL) (22). Results were considered significant with P < 0.05. Associations between both patient and parent IBD-KID scores and a number of independent variables were evaluated in bivariate or multivariate analyses. The independent variables were those listed in Table 1. t Tests and Pearson correlation coefficients were used for comparisons using dichotomous and continuous independent variables, respectively. Two backward multiple linear regression analyses were performed: 1 with patient knowledge score as the dependent variable and 1 with parent knowledge score as the dependent variable. Independent variables were the same as those used in univariate analyses. In each case, means were compared using a 1-way analysis of variance Tukey test. In terms of the analysis of the reliability phase, intraclass correlation coefficients were calculated for parents and patients separately, and for the entire group, to calculate Cronbach a for the entire group and for patients and parents separately.
RESULTS Testing of the IBD-KID Questionnaire Validity of the IBD-KID Questionnaire For the validity assessment, a total of 137 pairs of questionnaires were sent to patients with IBD and their parents,
315
Copyright 2014 by ESPGHAN and NASPGHAN. Unauthorized reproduction of this article is prohibited.
Haaland et al
JPGN
Volume 58, Number 3, March 2014
TABLE 1. A priori constructs to assess validity and discriminatory ability of the IBD-KID
Construct
Resulty
Statistic
Association of IBD-KID scores of patients with IBD with specific variables Scores should correlate positively with perceived Pearson correlation coefficient knowledge level Scores should correlate positively with parent scores Pearson correlation coefficient
Scores should correlate positively with disease duration Scores should correlate positively with patient age Patients who have completed grade 8 should score higher
Pearson correlation coefficient Pearson correlation coefficient t test
Patients whose parent has postsecondary education should score higher Patients with CCFC membership should score higher
t test
Patients who have a close relative with IBD should score higher
t test
Scores should not differ between male and female patients
t test
Scores should not differ between CD and UC patients
t test
Association of IBD parent IBD-KID scores with specific variables Scores should correlate positively with perceived knowledge level Scores should correlate positively with patient scores
t test
Pearson correlation coefficient Pearson correlation coefficient
Scores should correlate positively with disease duration Parents with postsecondary education should score higher
Pearson correlation coefficient t test
Parents with CCFC membership should score higher
t test
Parents who have a close relative with IBD should score higher IBD-KID mean scores and level of professional training IBD-KID scores should be as follows: pediatric residents > nurses > ward clerks
t test
IBD-KID mean scores among IBD and control populationsz IBD-KID scores should be as follows: IBD parents > Patients with IBD >control parents > control patients
Pearson coefficient ¼ 0.43 P < 0.001 Pearson coefficient ¼ 0.49 P < 0.001 P ¼ NS P ¼ NS >G8: 11.60 0.47; G8 10.85 0.60 P ¼ NS Yes: 11.52 0.45; no: 10.81 0.66 P ¼ NS Yes: 12.61 0.84; no: 10.89 0.40 P ¼ 0.025 Yes: 11.44 0.50; no: 11.19 0.53 P ¼ NS Male 11.1 3.7 (n ¼ 42); female 11.8 4.2 (n ¼ 36), P ¼ NS CD: 12.4 4.0 (n ¼ 46); UC 10.1 3.4 (n ¼ 32), P ¼ 0.01 Pearson coefficient ¼ 0.40 P < 0.001 Pearson coefficient ¼ 0.49 P < 0.001 P ¼ NS Yes: 15.45 0.44; no: 12.69 0.63 P < 0.001 Yes: 15.08 0.86; no: 14.39 0.42 P ¼ NS Yes: 14.61 0.55; no: 14.52 0.53 P ¼ NS
Tukey
Residents ¼ 19.08 0.84 (n ¼ 12) Nurses ¼ 15.79 0.64 (n ¼ 19) Ward clerks ¼ 9.24 1.07 (n ¼ 21) P < 0.001 except: P ¼ 0.03 for nurses vs ward clerks
Tukey
IBD parents ¼ 14.56 0.38 (n ¼ 99) Patients with IBD ¼ 11.29 0.37 (n ¼ 99) Control parents ¼ 7.43 0.71 (n ¼ 28) Control patients ¼ 4.82 0.63 (n ¼ 28) P < 0.001 except: P ¼ 0.02 control parents vs control patients
CCFC ¼ Crohn’s and Colitis Foundation of Canada; CD ¼ Crohn disease; IBD-KID ¼ Inflammatory Bowel Disease-Knowledge Inventory Device; NS ¼ not significant; UC ¼ ulcerative colitis. IBD patient and parent variables were also entered into separate backward multiple linear regression models (see text). y See text for results of multiple linear regression models. z Control comparisons were not adjusted for demographic variables (eg, age, education).
of which 99 pairs were returned, yielding a response rate of 72%. Table 2 describes the demographics and disease characteristics of the patient population. Of the 99 parents who responded, 87 (88%) were female and 67 (68%) had completed some form of postsecondary education. Wherein specific differences were predicted, bivariate analyses of patient scores and parent scores are presented in Table 1. Patients with Crohn disease (CD) scored higher than patients with ulcerative colitis (UC) (11.39 0.49 vs 10.35 0.54, P ¼ 0.035). This did not hold true for parents of patients with CD versus parents of patients with UC (P ¼ NS). IBD-KID scores
316
did not differ between male and female patients; however, female parents scored higher than male parents (14.93 0.38 vs 11.83 1.25; P ¼ 0.007). In multivariate analysis of patient IBD-KID scores, perceived knowledge and parent total score were significant (P < 0.001) and disease type bordered on significance (P ¼ 0.052) with patients with CD scoring higher. The model including these 3 variables yielded an adjusted R2 0.357 (P < 0.001). Significant variables in multivariate analysis of parent IBDKID scores were parent postsecondary education (P < 0.001), patient total (P < 0.001), perceived knowledge level (P ¼ 0.001), www.jpgn.org
Copyright 2014 by ESPGHAN and NASPGHAN. Unauthorized reproduction of this article is prohibited.
JPGN
Volume 58, Number 3, March 2014
Development of the IBD-KID
TABLE 2. Demographics of patients with IBD N ¼ 99 Sex Age (mean SD), y School grade Disease type Disease duration (mean SD), y CCFC membership Relative with IBD (%)
Male ¼ 52; female ¼ 47 14.6 0.2 Grade 8 ¼ 41 CD ¼ 59; UC ¼ 40 3.2 0.2 Yes ¼ 23; no ¼ 76 41 (41)
CCFC ¼ Crohn’s and Colitis Foundation of Canada; CD ¼ Crohn disease; IBD ¼ inflammatory bowel disease; SD ¼ standard deviation; UC ¼ ulcerative colitis. Relative with IBD indicated parent with first-degree relative having IBD.
and parent sex (women scored higher; P ¼ 0.021). The adjusted R2 for the model was 0.428 (P < 0.001). The response rates of the pediatric residents, nurses, and ward clerks were 67%, 63%, and 70%, respectively. A comparison of scores obtained by these 3 groups, as well as by parents of patients with IBD, patients with IBD, control parents, and control patients, is presented in Table 1.
Reliability of the IBD-KID Questionnaire Seventeen patients and parents completed 2 sets of questionnaires within 2 to 4 weeks. The intraclass correlation was 0.84 (95% confidence interval 0.70–0.92). The internal consistency was measured using Cronbach a. The Cronbach a was 0.75 (95% confidence interval 0.69–0.80).
Feasibility of the IBD-KID Questionnaire The Flesch Reading Ease score was 69.0, and the FleschKincaid Grade Level score was 6.3. Of the total questions answered, only 0.9% (42/4554) were completed improperly (eg, left blank, >1 answer circled). The values for patients and parents were 1.1% (26/2277) and 0.7% (16/2277), respectively.
Performance of pediatric patients with IBD and their parents on selected IBD-KID questions is presented in Table 3. A copy of the IBD-KID assessment is provided online, http://links.lww.com/ MPG/A275.
DISCUSSION This is the first IBD-specific knowledge questionnaire designed for and validated in a pediatric IBD population. This questionnaire was developed solely in a pediatric setting, and was established to provide a valid and reliable assessment of knowledge in these children and their parents. The questionnaire was validated using a number of methods. First, scores were predicted to correlate with several variables a priori. In univariate and regression analyses, as expected, patient and parent scores correlated with perceived knowledge level and with respective parent and patient scores. In univariate but not regression analyses, parent postsecondary education correlated with parent scores (17,23), but not patient scores. The association in univariate analysis was similar to the findings of Jones et al (6). In univariate but not regression analyses, CCFC membership correlated with patient, but not parent, score. Previous adult studies (6,7) have found that patients with membership in the Crohn’s and Colitis UK scored higher on knowledge questionnaires. This suggests that these individuals are more interested in learning about their disease and/or have greater exposure to educational material through their membership. In univariate analysis, the IBD-KID score was higher among patients with CD versus UC, and this bordered on significance in regression analysis (P ¼ 0.052). This finding was not predicted, but has been noted previously (6,23). This may relate to the fact that CD can run a more severe course than UC and that patients with CD report worse health-related quality of life than those with UC (24). Furthermore, CD has a greater likelihood of systemic manifestations, more extraintestinal manifestations, and higher rate of growth retardation (25) and is associated with more pain (26). This relation between level of knowledge and disease burden has been demonstrated in other disease entities. Among patients with type 1 diabetes mellitus, it was found that a knowledge deficit was associated with a lack of interest in disease information when feeling well (27).
TABLE 3. Performance of pediatric patients with IBD and their parents on selected IBD-KID questions No. correct responses, N ¼ 99 Concept No risk of IBD transmission The etiology of IBD is unknown IBD is familial Most patients with IBD have normal life expectancy Emotional stress and IBD flare-up Correct order of digestive tract IBD can affect other organs Risk of colon cancer in IBD No foods proven to cause IBD flare-ups Osteoporosis is related to IBD, steroids, and poor nutrition Herbal remedies can have adverse effects and interact with drugs Role of enteral nutrition (treatment and prevention of CD/growth) Role of surgery in CD and UC Role of salicylates and azathioprine in maintenance therapy
Patient
Parent
90 78 71 70 69 62 47 37 36 29 24 20 16 9
96 89 74 62 79 78 70 49 50 49 32 41 32 23
CD ¼ Crohn disease; IBD-KID ¼ Inflammatory Bowel Disease-Knowledge Inventory Device; UC ¼ ulcerative colitis.
www.jpgn.org
317
Copyright 2014 by ESPGHAN and NASPGHAN. Unauthorized reproduction of this article is prohibited.
Haaland et al Surprisingly, there was no association between knowledge score and days since diagnosis—as some previous adult asthma and diabetes literature has found—the longer the duration of disease, the better the disease knowledge (17,18). This finding did, however, confirm previous results among adult patients (7,28). It is possible that core information is obtained shortly after diagnosis, with minimal new information being attained subsequently or forgotten over time. There was no association between patient IBD-KID score and age, which was also the case with medication knowledge in youth with IBD (29) and similar to results in adult patients with rheumatoid arthritis (28); however, in a study of pediatric and young adult patients with asthma, age was the most significant predictor of asthma knowledge (15). There was also no association with school grade, which may be explained by the lack of association between knowledge and age. Another unpredicted finding in the present study was that mothers scored higher than fathers in univariate and regression analyses. Mothers were also far more likely than fathers to be the parent who chose to complete the questionnaire (87 vs 12 respondents). Similar results were found among the parents of patients with cystic fibrosis (16). These findings may be explained in part by literature that suggests that the impact of a chronically ill child is experienced more greatly by the mother than by the father and that fathers, more often than mothers, describe themselves as outside observers in terms of the management of their child’s illness (30). Further validation was achieved through demonstration that, as predicted, mean scores of pediatric residents, nurses, and ward clerks differed significantly. A comparable method of validation was used previously (7). Similarly, as hypothesized, significantly different scores were obtained from parents of patients with IBD, patients with IBD, control parents, and control patients. This ability to distinguish among groups based on hypothesized knowledge level is the strength of this device. The questionnaire also demonstrated excellent test-retest reliability (21). The Cronbach a of the IBD-KID (0.75) was lower than those of previous IBD knowledge questionnaires (0.84 obtained by Jones et al (6); 0.95 obtained by Eaden et al (7)). A value of >0.70 is recommended for comparison among groups of patients, and >0.90 is suggested for analyzing individual patient scores (31). Cronbach a is a useful measure, but it is not fully relevant when applied to a knowledge questionnaire because disease-related knowledge is not truly a single entity, but rather is multifaceted. For instance, it is quite reasonable that a given individual may know a lot about medication adverse effects, but be relatively uninformed about disease complications. The IBD-KID scored extremely well in terms of feasibility for both patients and parents. The Flesch Reading Ease score for the questionnaire is 69, which is considered to be easier to understand than ‘‘plain English.’’ The Flesch-Kincaid Grade Level score is grade 6. Because the questionnaire was targeted toward pediatric patients with IBD ages 10 to 17 and their parents, the ideal FleschKincaid Grade Level score for the questionnaire would thus be in the range of 4 to 5; however, IBD-related terms including ulcerative colitis, osteoporosis, colonoscopy, and drug names such as sulfasalazine and azathioprine were purposefully included in the questionnaire because it was believed that an understanding of such terms is an essential component of overall knowledge of IBD. Because the average number of syllables per word factors heavily into the Flesch Reading Ease score, the inclusion of these terms adversely affected this score. It is reassuring that patient score did not correlate with age or grade, suggesting that reading ability did not hinder performance among younger patients. This study revealed several important knowledge gaps among the Maritime Canadian pediatric patient population with
318
JPGN
Volume 58, Number 3, March 2014
IBD and their parents. For instance, only approximately half of the parents and approximately one-third of patients were aware that IBD, corticosteroid use, and poor nutrition could result in osteoporosis. In their assessment of knowledge in adult patients, Jones et al (6) found that only 45% of patients knew that steroids could cause osteoporosis. In the present study, just 35% of patients and 41% of parents were aware that herbal medications have the potential to interact with conventional drugs; however, 70% of parents but only 47% of patients were aware that IBD may affect organs other than the bowels. Eaden et al (7) found that 53% of adult patients were aware of this fact. Sixty-three percent of patients and 47% of adults either believed that removing certain foods from the diet would prevent flare-ups of IBD or were unsure. A possible limitation of this instrument may be the need to adapt or change the questionnaire over time. We also did not measure disease severity; it would be beneficial to include this variable in future studies. In a study of pediatric and young adult patients with asthma, it was found that more severe disease was positively related to knowledge (15). Lastly, because every parent– child pair completed the questionnaire, we cannot rule out the possibility that the child did not complete the assessment of their own accord, but rather was encouraged by the parent; however, parents and patients were instructed to complete the questionnaire separately and to check the ‘‘do not know’’ category when unsure of an answer, to reduce pressure. Using a pediatric-specific questionnaire should reduce bias by including questions that are relevant to the population, and by formatting/writing questions that maximize readability and ease of use for a young audience. Thus, although some bias is possible given the design of the present study, the methodology of questionnaire development also reduced other sources of bias. In conclusion, the IBD-KID is a valid and useful tool for the evaluation of IBD-related knowledge among pediatric patients with IBD and their parents. High levels of disease-related knowledge have been associated with improved disease outcomes (32,33). A recent study among adult patients found that greater IBD knowledge was associated with the use of more adaptive coping strategies (34). Furthermore, other adult reports suggest that providing IBD education reduces medication nonadherence (35,36) and that treatment adherence is positively associated with patient awareness of their disease and medications (37). The IBD-KID may also encourage self-management and aid in the transition of patients from pediatric to adult care by identifying and promoting awareness of areas of knowledge paucity. Future work with the pediatric population could entail carrying out an experimental study to look at the cause-and-effect relation between increasing disease knowledge and disease outcomes such as medication adherence and self-management. This study identified many important knowledge deficits in the study population. It has been reported that there exists a lack of medical knowledge in IBD transitioned and adolescent patients (38–40) and that many adolescent patients with IBD are unaware of the adverse effects of their IBD medications (29). Acknowledgments: The authors thank the hospital staff, patients, and their parents for their assistance with this study.
REFERENCES 1. Hughes DM, McLeod M, Garner B, et al. Controlled trial of a home and ambulatory program for asthmatic children. Pediatrics 1991;87: 54–61. 2. Couper JJ, Taylor J, Fotheringham MJ, et al. Failure to maintain the benefits of home-based intervention in adolescents with poorly controlled type 1 diabetes. Diabetes Care 1999;22:1933–7.
www.jpgn.org
Copyright 2014 by ESPGHAN and NASPGHAN. Unauthorized reproduction of this article is prohibited.
JPGN
Volume 58, Number 3, March 2014
3. Lorig K, FP, Regan C, Ung E, et al. A comparison of lay-taught and professional-taught arthritis self-management courses. J Rheumatol 1986;13:763–7. 4. Brown S. Effects of educational interventions in diabetes care: a metaanalysis of findings. Nurs Res 1988;37:223–30. 5. Kappelman MD, Rifas-Shiman SL, Kleinman K, et al. The prevalence and geographic distribution of Crohn’s disease and ulcerative colitis in the United States. Clin Gastroenterol Hepatol 2007;5:1424–9. 6. Jones SC, Gallacher B, Lobo AJ, et al. A patient knowledge questionnaire in inflammatory bowel disease. J Clin Gastroenterol 1993;17:21–4. 7. Eaden JA, Abrams K, Mayberry JF. The Crohn’s and Colitis Knowledge Score: a test for measuring patient knowledge in inflammatory bowel disease. Am J Gastroenterol 1999;94:3560–6. 8. Mansfield JC, Tanner AR, Bramble MG. Information for patients about inflammatory bowel disease. J R Coll Phys London 1997;31:184–7. 9. Eaden JA, Ward B, Smith H, et al. Are we telling patients enough? A pilot study to assess patient information needs in a gastroenterology outpatient department. Eur J Gastroenterol Hepatol 1998;10:63–7. 10. Hilsden RJ, Dunn C, Patten S, et al. Information needs and seeking behaviors of patients with inflammatory bowel disease. Gastroenterology 1998; Abstract G4078:A995. 11. Scholmerich J, Sedlak P, Hoppe-Seyler P, et al. The information needs and fears of patients with inflammatory bowel disease. Hepatogastroenterology 1987;34:182–5. 12. Moser G, Tillinger W, Sachs G, et al. Disease-related worries and concerns: a study on out-patients with inflammatory bowel disease. Eur J Gastroenterol Hepatol 1995;7:853–8. 13. Otley AR. Validation of the IMPACT Questionnaire for Measuring Health-Related Quality of Life in Paediatric Inflammatory Bowel Disease: Sensibility, Validity and Reliability Testing in a Paediatric Cohort [Dissertation] Toronto: University of Toronto; 2004. 14. Hubbard JPCW. Multiple Choice Examinations in Medicine. London: Henry Kimpton; 1961. 15. Barton C, Abramson M, Aroni R, et al. What determines knowledge of asthma among young people and their families? J Asthma 2002;39: 701–9. 16. Nolan T, Desmond K, Herlich R, et al. Knowledge of cystic fibrosis in patients and their parents. Pediatrics 1986;77:229–35. 17. Emtner M, Hedin A, Andersson M, et al. Impact of patient characteristics, education and knowledge on emergency room visits in patients with asthma and COPD: a descriptive and correlative study. BMC Pulm Med 2009;9:43. 18. Firestone DN, Jimenez-Briceno L, Reimann JO, et al. Predictors of diabetes-specific knowledge and treatment satisfaction among Costa Ricans. Diabetes Educ 2004;30:281–92. 19. Streiner DL, Norman GR. Health Measurement Scales: A Practical Guide to Their Development and Use. 2nd ed. New York: Oxford University Press; 1995. 20. Flesch RF. A new readability yardstick. J Appl Psychol 1948;32: 221–33. 21. McHorney CA, Ware JE, Lu JFR, et al. The MOS 36-Item Short-Form Health Survey (SF-36): III. Tests of Data quality, scaling assumptions, and reliability across diverse patient groups. Med Care 1994;32:40–66. 22. SPSS (for Windows) [computer program]. Version 10.0.5. Chicago, IL: SPSS Inc; 1999.
www.jpgn.org
Development of the IBD-KID
23. Subasinghe D, Wijekoon NS, Nawarathne NM, et al. Disease-related knowledge in inflammatory bowel disease: experience of a tertiary care centre in a developing country in South Asia. Singapore Med J 2014;51:484–9. 24. Larsson K, Loof L, Ronnblom A, et al. Quality of life for patients with exacerbation in inflammatory bowel disease and how they cope with disease activity. J Psychosom Res 2008;64:139–48. 25. Griffiths AM. Inflammatory bowel disease. Adolesc Med State Art Rev 1995;6:351–68. 26. Lix LM, Graff LA, Walker JR, et al. Longitudinal study of quality of life and psychological functioning for active, fluctuating, and inactive disease patterns in inflammatory bowel disease. Inflamm Bowel Dis 2008;14:1575–84. 27. Rankin D, Heller S, Lawton J. Understanding information and education gaps among people with type 1 diabetes: a qualitative investigation. Patient Educ Couns 2011;83:87–91. 28. Hennell SL, Brownsell C, Dawson JK. Development, validation and use of a patient knowledge questionnaire (PKQ) for patients with early rheumatoid arthritis. Rheumatology 2004;43:467–71. 29. Fishman LN, Houtman D, van Groningen J, et al. Medication knowledge: an initial step in self-management for youth with inflammatory bowel disease. J Pediatr Gastroenterol Nutr 2011;53:641–5. 30. Knafl K, Zoeller L. Childhood chronic illness: a comparison of mothers’ and fathers’ experiences. J Fam Nurs 2000;6:287–302. 31. Portney LG, Watkins MP. Statistical measures of reliability. In: Foundation of Clinical Research: Applications to Practice. Norwich, CT: Appleton and Lange; 1993. 32. Hampson SE, Skinner TC, Hart J, et al. Effects of educational and psychosocial interventions for adolescents with diabetes mellitus: a systematic review. Health Technol Assess 2001;5:1–79. 33. Loveman E, Cave C, Green C, et al. The clinical and cost-effectiveness of patient education models for diabetes: a systematic review and economic evaluation. Health Technol Assess 2003;7:1–190iii. 34. Moradkhani A, Kerwin L, Dudley-Brown S, et al. Disease-specific knowledge, coping, and adherence in patients with inflammatory bowel disease. Dig Dis Sci 2011;56:2972–7. 35. Waters BM, Jensen L, Fedorak RN. Effects of formal education for patients with inflammatory bowel disease: a randomized controlled trial. Can J Gastroenterol 2005;19:235–44. 36. Hawthorne AB, Rubin G, Ghosh S. Review article: medication nonadherence in ulcerative colitis: strategies to improve adherence with mesalazine and other maintenance therapies. Aliment Pharmacol Ther 2008;27:1157–66. ˇ erveny´ P, Bortlı´k M, Kubeˇna A, et al. Nonadherence in inflammatory 37. C bowel disease: results of factor analysis. Inflamm Bowel Dis 2007;13: 1244–9. 38. Hait EJ, Barendse ReM, Arnold JH, et al. Transition of adolescents with inflammatory bowel disease from pediatric to adult care: a survey of adult gastroenterologists. J Pediatr Gastroenterol Nutr 2009;48:61–5. 39. Fishman LN, Barendse ReM, Hait E, et al. Self-management of older adolescents with inflammatory bowel disease: a pilot study of behavior and knowledge as prelude to transition. Clin Pediatr 2010;49:1129–33. 40. Benchimol EI, Walters TD, Kaufman M, et al. Assessment of knowledge in adolescents with inflammatory bowel disease using a novel transition tool. Inflamm Bowel Dis 2011;17:1131–7.
319
Copyright 2014 by ESPGHAN and NASPGHAN. Unauthorized reproduction of this article is prohibited.
