G Model
JINJ-5862; No. of Pages 7 Injury, Int. J. Care Injured xxx (2014) xxx–xxx
Contents lists available at ScienceDirect
Injury journal homepage: www.elsevier.com/locate/injury
Measuring illness beliefs in patients with lower extremity injuries: Reliability and validity of the Dutch version of the Somatic Pre-Occupation and Coping questionnaire (SPOC-NL) Inge H.F. Reininga a,*, Sandra Brouwer b, Anita Dijkstra a, Jason W. Busse c,d,e, Shanil Ebrahim c,d,f,g, Klaus W. Wendt a, Mostafa el Moumni a a
Department of Trauma Surgery, University of Groningen, University Medical Center Groningen, Groningen, The Netherlands Department of Health Sciences, Division of Community & Occupational Medicine, University of Groningen, University Medical Center Groningen, Groningen, The Netherlands c Department of Anesthesia, McMaster University, Hamilton, Ontario, Canada d Department of Biostatistics and Clinical Epidemiology, McMaster University, Hamilton, Ontario, Canada e The Michael G. DeGroote Institute for Pain Research and Care, McMaster University, Hamilton, Ontario, Canada f Stanford Prevention Research Center, Department of Medicine, Stanford University, Stanford, USA g Department of Anaesthesia & Pain Medicine, The Hospital for Sick Children, Toronto, Ontario, Canada b
A R T I C L E I N F O
A B S T R A C T
Article history: Accepted 29 August 2014
Introduction: Positive coping strategies, illness perceptions and recovery expectations are associated with better clinical outcomes and earlier return to work after injuries. The Somatic Pre-Occupation and Coping (SPOC) questionnaire captures illness beliefs and coping towards recovery of physical function and return to work after surgical treatment of tibial shaft fractures. The aim of this study was to translate and culturally adapt the SPOC into Dutch (SPOC-NL) and evaluate its reliability and validity in patients with lower extremity injuries. Materials: The SPOC-NL contains four subscales: Somatic complaints, Coping, Energy, and Optimism. Patients treated for lower extremity injuries (N = 106) completed the SPOC-NL, Short Form-36 and Short Musculoskeletal Function Assessment (SMFA-NL) questionnaire, and reported their current work status and self-perceived work ability. To assess test–retest reliability, 56 patients completed the SPOC-NL for a second time two weeks after the first administration of the SPOC-NL. We calculated Cronbach’s Alpha, intraclass correlation coefficients (ICCs) and G coefficients to measure internal consistency and overall reliability, and used the Bland and Altman method to assess bias between test and retest SPOC-NL scores. To determine construct validity, we explored 16 a priori hypotheses regarding correlations between SPOC-NL scores and subscale scores and SF-36, SMFA-NL, work status and work ability. Results: Internal consistency was good to excellent, with Cronbach’s Alpha values ranging between 0.79 and 0.94 and G coefficients ranging between 0.77 and 0.95. Test–retest reliability was also good, since high ICCs (0.72–0.91) and G coefficients (0.82–0.94) were found. Construct validity of the SPOC-NL was good, as 75% of the predefined hypotheses were confirmed. Compared to participants who were on sick leave or receiving disability benefits, participants with a paid job had significantly higher scores on the total score and the subscales Somatic complaints and Energy of the SPOC-NL. Participants with high work ability also had significantly higher scores on the total score and all subscales than participants with low work ability. Discussion and conclusions: The SPOC-NL is a reliable and valid measure for the assessment of illness beliefs and coping towards recovery and it is strongly related to work status and self-perceived ability to work in patients with lower extremity injuries. ß 2014 Elsevier Ltd. All rights reserved.
Keywords: Injury Lower extremity Illness beliefs Recovery expectations Coping Cross-cultural adaptation Clinimetric properties Return to work
* Corresponding author at: University Medical Center Groningen, Department of Trauma Surgery, P.O. Box 30.001, 9700 RB Groningen, The Netherlands. Tel.: +31 503610218. E-mail address: [email protected] (Inge H.F. Reininga). http://dx.doi.org/10.1016/j.injury.2014.08.042 0020–1383/ß 2014 Elsevier Ltd. All rights reserved.
Please cite this article in press as: Reininga IHF, et al. Measuring illness beliefs in patients with lower extremity injuries: Reliability and validity of the Dutch version of the Somatic Pre-Occupation and Coping questionnaire (SPOC-NL). Injury (2014), http://dx.doi.org/ 10.1016/j.injury.2014.08.042
G Model
JINJ-5862; No. of Pages 7 2
I.H.F. Reininga et al. / Injury, Int. J. Care Injured xxx (2014) xxx–xxx
Introduction
Application of the SPOC-NL to Dutch lower limb trauma patients
Traumatic injuries are a leading contributor to the global burden of disability for all age groups below age 60 [1], and have a major economic burden on society, since the frequency of traumatic injuries is high among individuals of working age [2,3]. It is difficult to predict from injury characteristics alone whether, and how quickly, patients will return to work (RTW) [4]. Both injury severity and type of injury are associated with resuming employment following injury [5,6]; however, the ability to RTW is more strongly associated with psychosocial factors [7]. Studies have demonstrated that positive coping strategies, illness perceptions and recovery expectations, are associated with better clinical outcomes and earlier RTW [8–11]. The Somatic Pre-Occupation and Coping (SPOC) questionnaire is a validated measure of illness beliefs in patients with operatively managed tibial shaft fractures. Scores on the SPOC obtained 6 weeks postoperatively were a more powerful predictor of functional recovery and RTW at 1 year following surgery than age, gender or fracture type [12]. Whether these results are also valid for patients with other types of injuries is unknown. Further, the SPOC is currently available only in English, and translated versions of the SPOC to different languages are required for cross-cultural comparisons. The aims of this study were to translate and culturally adapt the SPOC into Dutch, and evaluate the reliability and validity of the Dutch version of the SPOC in patients with lower extremity injuries.
Participants Eligible patients were between 18 and 75 years old, treated for various musculoskeletal injuries of the lower limb at the department of Trauma Surgery of [BLINDED 3], and underwent surgery between six weeks and one year prior to the start of the study (November 1st 2012). The following patient were excluded: (1) patients with head trauma or amputation of a lower limb; (2) patients with co-morbidities such as a neuromuscular disease, cancer, or a cardiovascular disease with an active episode in the three months previous to the start of this study; (3) patients with serious psychiatric or cognitive disorders; and (4) patients who were unable to understand written Dutch. Eligible participants were contacted by mail, and those who consented to participate were asked to complete all study questionnaires at home. It was our opinion that patients who were surgically treated at least three months before the start of the study were likely to be in a stable state and we asked these individuals to complete the SPOC-NL questionnaire for a second time after an interval of two weeks to explore test–retest reliability. We confirmed our assumption by having these same patients provide a Global Rating of Change (GRC) score to quantify the improvement or deterioration in their health status during the previous two weeks. The GRC was scored on a 6-item Likert scale, ranging from much improved to much deteriorated. Participants who reported being much improved or much deteriorated on the GRC were excluded from our test–retest analysis. Measures
Materials The study was divided into two stages. First, the SPOC was translated into Dutch according to a standardized procedure [13]. Second, the translated version was tested for clinimetric quality in a prospective study. The Institutional Review Board of [BLINDED 1] approved this study. Translation process The translation and cross-cultural adaptation of the SPOC into Dutch was performed according to the Guidelines proposed by Beaton et al. [13]. This process is divided into five stages. In the first stage, two bilingual translators with Dutch as their first language independently translated the SPOC into Dutch. In the second stage, discrepancies between translations were resolved by discussion to produce a consensus questionnaire. In the third stage two bilingual translators whose native tongue was English independently translated the Dutch SPOC back into English. These translators were blind to the original version of the SPOC, and were neither aware of the concepts explored in the SPOC. With this backtranslation process, the content validity of the questionnaire is verified to ensure that the translated version reflects the same item content as the original version. These two back-translations were reviewed in stage 4 by an expert committee, which consisted of a trauma surgeon, a human movement scientist and a clinical epidemiologist. Discrepancies were resolved by discussion to achieve consensus. This resulted in a pre-final Dutch version of the SPOC (SPOC-NL). To evaluate the comprehensibility of the pre-final version of the SPOC-NL a pre-test was performed in a group of 20 patients who had an appointment at the outpatient clinic of the department of Trauma Surgery [BLINDED 2] (stage 5). After completion of the pre-final version of the SPOC-NL, a short interview was conducted with each participant to explore their impressions regarding the questionnaire’s usability, applicability and comprehension of the questionnaire.
SPOC. The Somatic Pre-Occupation and Coping (SPOC) questionnaire is a 27-item instrument that captures illness beliefs and coping with respect to injuries. The SPOC questionnaire is divided into four subscales: Somatic complaints (10 items), Coping (6 items), Energy (7 items), and Optimism (4 items). Each item is scored on a 7-point Likert scale ranging from 0 to 6 [12]. We calculated the total SPOC-NL score, and a score per subscale, for each patient with higher scores indicating more positive illness beliefs and better coping. SF-36. The 36-item Short Form Health Survey (SF-36) is a generic questionnaire that measures health-related quality of life, and a validated Dutch version is available [14]. The SF-36 is composed of 36 questions, organized into 8 multi-item scales: Physical functioning, Role limitations due to physical health problems, Bodily pain, General health perceptions, Vitality, Social functioning, Role limitations due to emotional problems, and General mental health. The raw score on each subscale was transformed to a 100-point scale, with higher scores indicating a better health status. SMFA-NL. The Short Musculoskeletal Function Assessment (SMFA) questionnaire consists of 46 items and is designed to measure health status of people who are suffering from a broad range of musculoskeletal disorders or injuries. The original SMFA consists of two indices: the Dysfunction Index and the Bother Index [15]. The Dutch version of the SMFA (SMFA-NL) consists of the same two indices, which have been reduced to four subscales: Lowerextremity dysfunction, Upper-extremity dysfunction, Problems with daily activities and Mental and emotional problems [16]. The score on each subscale was transformed to a 100-point scale, with higher scores indicating better function. Work status and ability to work. Participants were asked to report their current work status. The response to this question consisted
Please cite this article in press as: Reininga IHF, et al. Measuring illness beliefs in patients with lower extremity injuries: Reliability and validity of the Dutch version of the Somatic Pre-Occupation and Coping questionnaire (SPOC-NL). Injury (2014), http://dx.doi.org/ 10.1016/j.injury.2014.08.042
G Model
JINJ-5862; No. of Pages 7 I.H.F. Reininga et al. / Injury, Int. J. Care Injured xxx (2014) xxx–xxx
of five options: (1) attached to paid employment; (2) attached to paid employment but currently on sick leave; (3) unemployed; (4) unemployed and receiving disability benefits; and (5) retired. Participants were also asked to indicate their current ability to work on a five-point Likert scale, ranging from very high to very low. Reliability We assessed both the internal consistency and test–retest reliability of the SPOC-NL. Internal consistency is a measure of the extent to which items in a questionnaire’s (sub)scale are correlated (homogeneous), thus measuring the same concept [17]. Test– retest reliability refers to the degree to which repeated measurements in stable persons (test–retest) provide the same answers [17]. Test–retest reliability consists of two parts: agreement and reliability. Agreement concerns the extent to which the scores on repeated measures are close to each other (absolute measurement error). Reliability concerns the extent to which participants can be distinguished from each other despite measurement errors (relative measurement error) [17]. Construct validity Validity of the SPOC-NL was expressed in terms of construct validity. This refers to the extent to which scores on a particular measure relate to other measures, consistent with theoretically derived hypotheses concerning the constructs that are being measured [17]. Construct validity of the SPOC-NL was determined by evaluating the relationship of the SPOC-NL with subscales of both the SF-36 and the SMFA-NL. According to the guidelines proposed by the COSMIN initiative [18], we formulated sixteen hypotheses about the magnitude of relationships between the SPOC-NL and the corresponding subscales of the SF-36 and the SMFA-NL, current work status and patient’s reported ability to work (Table 1). Not all subscales of the SF-36 and the SMFA-NL were used to determine construct validity of the SPOC-NL. As proposed by Terwee et al. [17], we considered the SPOC-NL to have a good construct validity if we were able to confirm at least 75% of the hypotheses. Statistical analysis A sample size of at least 100 is considered excellent for studies regarding measurement properties of questionnaires, and a sample size of 50 is considered adequate for determining test–retest reliability [19]. Hence, we planned a sample size of at least 100 participants for assessing the construct validity of the SPOCNL, and a sample size of at least 50 for establishing the test–retest reliability of the SPOC-NL. Reliability We assessed internal consistency of the SPOC-NL with Cronbach’s alpha and G coefficients using the SPOC-NL total and subscale scores from the first administration. Additionally, we performed a sensitivity analysis for which the internal consistency across two time points was assessed, for which G coefficients for internal consistency were calculated based on the test–retest reliability data (data of the first and second administration of the SPOC-NL). For these G coefficients the variance due to participants and administration of the SPOC-NL (first or second administration) was held constant. Terwee and colleagues have suggested that internal consistency measures should be between 0.7 and 0.95 [17]. For test–retest reliability, we calculated the intraclass correlation coefficient (ICC) with corresponding 95% confidence intervals (CI) for each subscale of the SPOC-NL using data from the first and second administration of the SPOC-NL. We used an ICC two-way random effects model and both subject and time
3
effects were taken as random [20]. ICCs above 0.70 are generally considered to be good [17]. We also employed Generalizability Theory (G theory) to calculate our reliability coefficients [21]. In comparison to classical test theory, G theory allows for simultaneous estimation of the magnitude of multiple sources of measurement error (e.g., items, raters, time points). If the measurement errors associated with the different sources of error in the study interact with one another, G coefficients may be markedly different from classical test theory reliability estimates. Test–retest reliability considers time as the source of error [22]. Therefore we calculated G coefficients for internal consistency across time (i.e., variance due to both items and time), and test–retest reliability that accounted for variance arising from time and items (i.e., variance due to participants and items was held constant). G coefficients above 0.70 are recommended when the measure is used for research [23]. Additionally, the method of Bland and Altman was used to assess absolute agreement [24]. We calculated the mean difference between the first and second administration of the SPOC-NL with a 95% CI. Zero lying within the 95% CI of the mean difference was considered a criterion for absolute agreement. Consequently, when zero lies outside the 95% CI, a bias in the measurements is indicated. The 95% limits of agreement (LOA) were also calculated. LOA are formulated as the mean difference 1.96 SDdiff, where SDdiff is the standard deviation of the mean difference between the first and second administration of the SPOC-NL [24].
Table 1 Predefined hypotheses and the confirmation or rejection of the hypotheses. 1 2 3 4
5
6 7
8
9 10 11
12
13 14 15
16
A correlation of 0.60 between SPOC-NL subscale Somatic complaints and SF-36 subscale Physical functioning A correlation of 0.60 between SPOC-NL subscale Somatic complaints and SF-36 subscale Bodily pain A correlation of 0.60 between SPOC-NL subscale Somatic complaints and SMFA-NL Function Index A correlation of 0.60 between SPOC-NL subscale Somatic complaints and SMFA-NL subscale Lower extremity dysfunction A correlation of 0.50 between SPOC-NL subscale Somatic complaints and SF-36 subscale Role limitations due to physical health problems A correlation of 0.50 between SPOC-NL subscale Somatic complaints and SMFA-NL Bother Index A correlation of 0.50 between SPOC-NL subscale Somatic complaints and SMFA-NL subscale Problems with daily activities A correlation of 0.60 between SPOC-NL subscale Coping and SF-36 subscale Role limitations due to emotional problems A correlation of 0.60 between SPOC-NL subscale Coping and SF-36 subscale General mental health A correlation of 0.60 between SPOC-NL subscale Coping and SMFA-NL subscale Mental and emotional problems A correlation of 0.50 between SPOC-NL subscale Energy and SF-36 subscale Role limitations due to emotional problems A correlation of 0.50 between SPOC-NL subscale Energy and SF-36 subscale Role limitations due to physical health problems A correlation of 0.50 between SPOC-NL subscale Energy and SMFA-NL subscale Mental and emotional problems A correlation of 0.50 between SPOC-NL subscale Optimism and SF-36 subscale General health perceptions Participants who had a paid job have significantly higher scores on the total scores and the subscales of the SPOC-NL compared to participants who received disability benefits or who were on sick leave Participants with a high work ability have significantly higher scores on the total scores and the subscales of the SPOC-NL compared to participants with a low work ability
+ = hypothesis is confirmed;
+ + + +
+
+ +
+
+
+
+
+
= hypothesis is rejected.
Please cite this article in press as: Reininga IHF, et al. Measuring illness beliefs in patients with lower extremity injuries: Reliability and validity of the Dutch version of the Somatic Pre-Occupation and Coping questionnaire (SPOC-NL). Injury (2014), http://dx.doi.org/ 10.1016/j.injury.2014.08.042
G Model
JINJ-5862; No. of Pages 7 I.H.F. Reininga et al. / Injury, Int. J. Care Injured xxx (2014) xxx–xxx
4
Construct validity We calculated Pearson’s Correlation coefficients between the total score and the subscales of the SPOC-NL and the respective subscales of the SF-36 and the SMFA-NL. Pearson’s Correlation coefficient was interpreted according to Domholdt: 0.00–0.25 = little if any correlation, 0.26–0.49 = weak correlation, 0.50– 0.69 = moderate correlation, 0.70–0.89 = strong correlation, 0.90–1.00 = very strong correlation [25]. We performed a One-way Analysis of Variance (ANOVA) with post hoc comparisons with Bonferroni corrections for multiple testing determine whether participants who were working in paid employment had significantly higher scores on the SPOC-NL compared to participants who were receiving disability benefits or who were on sick leave, and compared to participants who were unemployed. Participants who were retired were not included in this analysis. We also performed a One-way ANOVA with post hoc comparisons with Bonferroni corrections for multiple testing to assess whether working participants with a high work had higher scores on the SPOC-NL, compared to participants with a moderate or low work ability. For that, work ability categories very high and high, and very low and low were merged to three categories (high, moderate and low work ability). Participants who were on sick leave, disability benefits or retired were excluded from these analyses. Non-response analysis A non-response analysis was performed, using the Chi-square, Student’s T-Test and Mann–Whitney U test, to determine whether there were differences in characteristics (gender, age and time interval between surgery and sending out the questionnaire) of the participants who did or did not respond to the SPOC-NL questionnaire. We used IBM SPSS Statistics for Windows (Version 20.0, Armonk, NY: IBM Corp.) and G-String_IV software (version 6.2.1.2) for all statistical analyses. A p-value of
JINJ-5862; No. of Pages 7 Injury, Int. J. Care Injured xxx (2014) xxx–xxx
Contents lists available at ScienceDirect
Injury journal homepage: www.elsevier.com/locate/injury
Measuring illness beliefs in patients with lower extremity injuries: Reliability and validity of the Dutch version of the Somatic Pre-Occupation and Coping questionnaire (SPOC-NL) Inge H.F. Reininga a,*, Sandra Brouwer b, Anita Dijkstra a, Jason W. Busse c,d,e, Shanil Ebrahim c,d,f,g, Klaus W. Wendt a, Mostafa el Moumni a a
Department of Trauma Surgery, University of Groningen, University Medical Center Groningen, Groningen, The Netherlands Department of Health Sciences, Division of Community & Occupational Medicine, University of Groningen, University Medical Center Groningen, Groningen, The Netherlands c Department of Anesthesia, McMaster University, Hamilton, Ontario, Canada d Department of Biostatistics and Clinical Epidemiology, McMaster University, Hamilton, Ontario, Canada e The Michael G. DeGroote Institute for Pain Research and Care, McMaster University, Hamilton, Ontario, Canada f Stanford Prevention Research Center, Department of Medicine, Stanford University, Stanford, USA g Department of Anaesthesia & Pain Medicine, The Hospital for Sick Children, Toronto, Ontario, Canada b
A R T I C L E I N F O
A B S T R A C T
Article history: Accepted 29 August 2014
Introduction: Positive coping strategies, illness perceptions and recovery expectations are associated with better clinical outcomes and earlier return to work after injuries. The Somatic Pre-Occupation and Coping (SPOC) questionnaire captures illness beliefs and coping towards recovery of physical function and return to work after surgical treatment of tibial shaft fractures. The aim of this study was to translate and culturally adapt the SPOC into Dutch (SPOC-NL) and evaluate its reliability and validity in patients with lower extremity injuries. Materials: The SPOC-NL contains four subscales: Somatic complaints, Coping, Energy, and Optimism. Patients treated for lower extremity injuries (N = 106) completed the SPOC-NL, Short Form-36 and Short Musculoskeletal Function Assessment (SMFA-NL) questionnaire, and reported their current work status and self-perceived work ability. To assess test–retest reliability, 56 patients completed the SPOC-NL for a second time two weeks after the first administration of the SPOC-NL. We calculated Cronbach’s Alpha, intraclass correlation coefficients (ICCs) and G coefficients to measure internal consistency and overall reliability, and used the Bland and Altman method to assess bias between test and retest SPOC-NL scores. To determine construct validity, we explored 16 a priori hypotheses regarding correlations between SPOC-NL scores and subscale scores and SF-36, SMFA-NL, work status and work ability. Results: Internal consistency was good to excellent, with Cronbach’s Alpha values ranging between 0.79 and 0.94 and G coefficients ranging between 0.77 and 0.95. Test–retest reliability was also good, since high ICCs (0.72–0.91) and G coefficients (0.82–0.94) were found. Construct validity of the SPOC-NL was good, as 75% of the predefined hypotheses were confirmed. Compared to participants who were on sick leave or receiving disability benefits, participants with a paid job had significantly higher scores on the total score and the subscales Somatic complaints and Energy of the SPOC-NL. Participants with high work ability also had significantly higher scores on the total score and all subscales than participants with low work ability. Discussion and conclusions: The SPOC-NL is a reliable and valid measure for the assessment of illness beliefs and coping towards recovery and it is strongly related to work status and self-perceived ability to work in patients with lower extremity injuries. ß 2014 Elsevier Ltd. All rights reserved.
Keywords: Injury Lower extremity Illness beliefs Recovery expectations Coping Cross-cultural adaptation Clinimetric properties Return to work
* Corresponding author at: University Medical Center Groningen, Department of Trauma Surgery, P.O. Box 30.001, 9700 RB Groningen, The Netherlands. Tel.: +31 503610218. E-mail address: [email protected] (Inge H.F. Reininga). http://dx.doi.org/10.1016/j.injury.2014.08.042 0020–1383/ß 2014 Elsevier Ltd. All rights reserved.
Please cite this article in press as: Reininga IHF, et al. Measuring illness beliefs in patients with lower extremity injuries: Reliability and validity of the Dutch version of the Somatic Pre-Occupation and Coping questionnaire (SPOC-NL). Injury (2014), http://dx.doi.org/ 10.1016/j.injury.2014.08.042
G Model
JINJ-5862; No. of Pages 7 2
I.H.F. Reininga et al. / Injury, Int. J. Care Injured xxx (2014) xxx–xxx
Introduction
Application of the SPOC-NL to Dutch lower limb trauma patients
Traumatic injuries are a leading contributor to the global burden of disability for all age groups below age 60 [1], and have a major economic burden on society, since the frequency of traumatic injuries is high among individuals of working age [2,3]. It is difficult to predict from injury characteristics alone whether, and how quickly, patients will return to work (RTW) [4]. Both injury severity and type of injury are associated with resuming employment following injury [5,6]; however, the ability to RTW is more strongly associated with psychosocial factors [7]. Studies have demonstrated that positive coping strategies, illness perceptions and recovery expectations, are associated with better clinical outcomes and earlier RTW [8–11]. The Somatic Pre-Occupation and Coping (SPOC) questionnaire is a validated measure of illness beliefs in patients with operatively managed tibial shaft fractures. Scores on the SPOC obtained 6 weeks postoperatively were a more powerful predictor of functional recovery and RTW at 1 year following surgery than age, gender or fracture type [12]. Whether these results are also valid for patients with other types of injuries is unknown. Further, the SPOC is currently available only in English, and translated versions of the SPOC to different languages are required for cross-cultural comparisons. The aims of this study were to translate and culturally adapt the SPOC into Dutch, and evaluate the reliability and validity of the Dutch version of the SPOC in patients with lower extremity injuries.
Participants Eligible patients were between 18 and 75 years old, treated for various musculoskeletal injuries of the lower limb at the department of Trauma Surgery of [BLINDED 3], and underwent surgery between six weeks and one year prior to the start of the study (November 1st 2012). The following patient were excluded: (1) patients with head trauma or amputation of a lower limb; (2) patients with co-morbidities such as a neuromuscular disease, cancer, or a cardiovascular disease with an active episode in the three months previous to the start of this study; (3) patients with serious psychiatric or cognitive disorders; and (4) patients who were unable to understand written Dutch. Eligible participants were contacted by mail, and those who consented to participate were asked to complete all study questionnaires at home. It was our opinion that patients who were surgically treated at least three months before the start of the study were likely to be in a stable state and we asked these individuals to complete the SPOC-NL questionnaire for a second time after an interval of two weeks to explore test–retest reliability. We confirmed our assumption by having these same patients provide a Global Rating of Change (GRC) score to quantify the improvement or deterioration in their health status during the previous two weeks. The GRC was scored on a 6-item Likert scale, ranging from much improved to much deteriorated. Participants who reported being much improved or much deteriorated on the GRC were excluded from our test–retest analysis. Measures
Materials The study was divided into two stages. First, the SPOC was translated into Dutch according to a standardized procedure [13]. Second, the translated version was tested for clinimetric quality in a prospective study. The Institutional Review Board of [BLINDED 1] approved this study. Translation process The translation and cross-cultural adaptation of the SPOC into Dutch was performed according to the Guidelines proposed by Beaton et al. [13]. This process is divided into five stages. In the first stage, two bilingual translators with Dutch as their first language independently translated the SPOC into Dutch. In the second stage, discrepancies between translations were resolved by discussion to produce a consensus questionnaire. In the third stage two bilingual translators whose native tongue was English independently translated the Dutch SPOC back into English. These translators were blind to the original version of the SPOC, and were neither aware of the concepts explored in the SPOC. With this backtranslation process, the content validity of the questionnaire is verified to ensure that the translated version reflects the same item content as the original version. These two back-translations were reviewed in stage 4 by an expert committee, which consisted of a trauma surgeon, a human movement scientist and a clinical epidemiologist. Discrepancies were resolved by discussion to achieve consensus. This resulted in a pre-final Dutch version of the SPOC (SPOC-NL). To evaluate the comprehensibility of the pre-final version of the SPOC-NL a pre-test was performed in a group of 20 patients who had an appointment at the outpatient clinic of the department of Trauma Surgery [BLINDED 2] (stage 5). After completion of the pre-final version of the SPOC-NL, a short interview was conducted with each participant to explore their impressions regarding the questionnaire’s usability, applicability and comprehension of the questionnaire.
SPOC. The Somatic Pre-Occupation and Coping (SPOC) questionnaire is a 27-item instrument that captures illness beliefs and coping with respect to injuries. The SPOC questionnaire is divided into four subscales: Somatic complaints (10 items), Coping (6 items), Energy (7 items), and Optimism (4 items). Each item is scored on a 7-point Likert scale ranging from 0 to 6 [12]. We calculated the total SPOC-NL score, and a score per subscale, for each patient with higher scores indicating more positive illness beliefs and better coping. SF-36. The 36-item Short Form Health Survey (SF-36) is a generic questionnaire that measures health-related quality of life, and a validated Dutch version is available [14]. The SF-36 is composed of 36 questions, organized into 8 multi-item scales: Physical functioning, Role limitations due to physical health problems, Bodily pain, General health perceptions, Vitality, Social functioning, Role limitations due to emotional problems, and General mental health. The raw score on each subscale was transformed to a 100-point scale, with higher scores indicating a better health status. SMFA-NL. The Short Musculoskeletal Function Assessment (SMFA) questionnaire consists of 46 items and is designed to measure health status of people who are suffering from a broad range of musculoskeletal disorders or injuries. The original SMFA consists of two indices: the Dysfunction Index and the Bother Index [15]. The Dutch version of the SMFA (SMFA-NL) consists of the same two indices, which have been reduced to four subscales: Lowerextremity dysfunction, Upper-extremity dysfunction, Problems with daily activities and Mental and emotional problems [16]. The score on each subscale was transformed to a 100-point scale, with higher scores indicating better function. Work status and ability to work. Participants were asked to report their current work status. The response to this question consisted
Please cite this article in press as: Reininga IHF, et al. Measuring illness beliefs in patients with lower extremity injuries: Reliability and validity of the Dutch version of the Somatic Pre-Occupation and Coping questionnaire (SPOC-NL). Injury (2014), http://dx.doi.org/ 10.1016/j.injury.2014.08.042
G Model
JINJ-5862; No. of Pages 7 I.H.F. Reininga et al. / Injury, Int. J. Care Injured xxx (2014) xxx–xxx
of five options: (1) attached to paid employment; (2) attached to paid employment but currently on sick leave; (3) unemployed; (4) unemployed and receiving disability benefits; and (5) retired. Participants were also asked to indicate their current ability to work on a five-point Likert scale, ranging from very high to very low. Reliability We assessed both the internal consistency and test–retest reliability of the SPOC-NL. Internal consistency is a measure of the extent to which items in a questionnaire’s (sub)scale are correlated (homogeneous), thus measuring the same concept [17]. Test– retest reliability refers to the degree to which repeated measurements in stable persons (test–retest) provide the same answers [17]. Test–retest reliability consists of two parts: agreement and reliability. Agreement concerns the extent to which the scores on repeated measures are close to each other (absolute measurement error). Reliability concerns the extent to which participants can be distinguished from each other despite measurement errors (relative measurement error) [17]. Construct validity Validity of the SPOC-NL was expressed in terms of construct validity. This refers to the extent to which scores on a particular measure relate to other measures, consistent with theoretically derived hypotheses concerning the constructs that are being measured [17]. Construct validity of the SPOC-NL was determined by evaluating the relationship of the SPOC-NL with subscales of both the SF-36 and the SMFA-NL. According to the guidelines proposed by the COSMIN initiative [18], we formulated sixteen hypotheses about the magnitude of relationships between the SPOC-NL and the corresponding subscales of the SF-36 and the SMFA-NL, current work status and patient’s reported ability to work (Table 1). Not all subscales of the SF-36 and the SMFA-NL were used to determine construct validity of the SPOC-NL. As proposed by Terwee et al. [17], we considered the SPOC-NL to have a good construct validity if we were able to confirm at least 75% of the hypotheses. Statistical analysis A sample size of at least 100 is considered excellent for studies regarding measurement properties of questionnaires, and a sample size of 50 is considered adequate for determining test–retest reliability [19]. Hence, we planned a sample size of at least 100 participants for assessing the construct validity of the SPOCNL, and a sample size of at least 50 for establishing the test–retest reliability of the SPOC-NL. Reliability We assessed internal consistency of the SPOC-NL with Cronbach’s alpha and G coefficients using the SPOC-NL total and subscale scores from the first administration. Additionally, we performed a sensitivity analysis for which the internal consistency across two time points was assessed, for which G coefficients for internal consistency were calculated based on the test–retest reliability data (data of the first and second administration of the SPOC-NL). For these G coefficients the variance due to participants and administration of the SPOC-NL (first or second administration) was held constant. Terwee and colleagues have suggested that internal consistency measures should be between 0.7 and 0.95 [17]. For test–retest reliability, we calculated the intraclass correlation coefficient (ICC) with corresponding 95% confidence intervals (CI) for each subscale of the SPOC-NL using data from the first and second administration of the SPOC-NL. We used an ICC two-way random effects model and both subject and time
3
effects were taken as random [20]. ICCs above 0.70 are generally considered to be good [17]. We also employed Generalizability Theory (G theory) to calculate our reliability coefficients [21]. In comparison to classical test theory, G theory allows for simultaneous estimation of the magnitude of multiple sources of measurement error (e.g., items, raters, time points). If the measurement errors associated with the different sources of error in the study interact with one another, G coefficients may be markedly different from classical test theory reliability estimates. Test–retest reliability considers time as the source of error [22]. Therefore we calculated G coefficients for internal consistency across time (i.e., variance due to both items and time), and test–retest reliability that accounted for variance arising from time and items (i.e., variance due to participants and items was held constant). G coefficients above 0.70 are recommended when the measure is used for research [23]. Additionally, the method of Bland and Altman was used to assess absolute agreement [24]. We calculated the mean difference between the first and second administration of the SPOC-NL with a 95% CI. Zero lying within the 95% CI of the mean difference was considered a criterion for absolute agreement. Consequently, when zero lies outside the 95% CI, a bias in the measurements is indicated. The 95% limits of agreement (LOA) were also calculated. LOA are formulated as the mean difference 1.96 SDdiff, where SDdiff is the standard deviation of the mean difference between the first and second administration of the SPOC-NL [24].
Table 1 Predefined hypotheses and the confirmation or rejection of the hypotheses. 1 2 3 4
5
6 7
8
9 10 11
12
13 14 15
16
A correlation of 0.60 between SPOC-NL subscale Somatic complaints and SF-36 subscale Physical functioning A correlation of 0.60 between SPOC-NL subscale Somatic complaints and SF-36 subscale Bodily pain A correlation of 0.60 between SPOC-NL subscale Somatic complaints and SMFA-NL Function Index A correlation of 0.60 between SPOC-NL subscale Somatic complaints and SMFA-NL subscale Lower extremity dysfunction A correlation of 0.50 between SPOC-NL subscale Somatic complaints and SF-36 subscale Role limitations due to physical health problems A correlation of 0.50 between SPOC-NL subscale Somatic complaints and SMFA-NL Bother Index A correlation of 0.50 between SPOC-NL subscale Somatic complaints and SMFA-NL subscale Problems with daily activities A correlation of 0.60 between SPOC-NL subscale Coping and SF-36 subscale Role limitations due to emotional problems A correlation of 0.60 between SPOC-NL subscale Coping and SF-36 subscale General mental health A correlation of 0.60 between SPOC-NL subscale Coping and SMFA-NL subscale Mental and emotional problems A correlation of 0.50 between SPOC-NL subscale Energy and SF-36 subscale Role limitations due to emotional problems A correlation of 0.50 between SPOC-NL subscale Energy and SF-36 subscale Role limitations due to physical health problems A correlation of 0.50 between SPOC-NL subscale Energy and SMFA-NL subscale Mental and emotional problems A correlation of 0.50 between SPOC-NL subscale Optimism and SF-36 subscale General health perceptions Participants who had a paid job have significantly higher scores on the total scores and the subscales of the SPOC-NL compared to participants who received disability benefits or who were on sick leave Participants with a high work ability have significantly higher scores on the total scores and the subscales of the SPOC-NL compared to participants with a low work ability
+ = hypothesis is confirmed;
+ + + +
+
+ +
+
+
+
+
+
= hypothesis is rejected.
Please cite this article in press as: Reininga IHF, et al. Measuring illness beliefs in patients with lower extremity injuries: Reliability and validity of the Dutch version of the Somatic Pre-Occupation and Coping questionnaire (SPOC-NL). Injury (2014), http://dx.doi.org/ 10.1016/j.injury.2014.08.042
G Model
JINJ-5862; No. of Pages 7 I.H.F. Reininga et al. / Injury, Int. J. Care Injured xxx (2014) xxx–xxx
4
Construct validity We calculated Pearson’s Correlation coefficients between the total score and the subscales of the SPOC-NL and the respective subscales of the SF-36 and the SMFA-NL. Pearson’s Correlation coefficient was interpreted according to Domholdt: 0.00–0.25 = little if any correlation, 0.26–0.49 = weak correlation, 0.50– 0.69 = moderate correlation, 0.70–0.89 = strong correlation, 0.90–1.00 = very strong correlation [25]. We performed a One-way Analysis of Variance (ANOVA) with post hoc comparisons with Bonferroni corrections for multiple testing determine whether participants who were working in paid employment had significantly higher scores on the SPOC-NL compared to participants who were receiving disability benefits or who were on sick leave, and compared to participants who were unemployed. Participants who were retired were not included in this analysis. We also performed a One-way ANOVA with post hoc comparisons with Bonferroni corrections for multiple testing to assess whether working participants with a high work had higher scores on the SPOC-NL, compared to participants with a moderate or low work ability. For that, work ability categories very high and high, and very low and low were merged to three categories (high, moderate and low work ability). Participants who were on sick leave, disability benefits or retired were excluded from these analyses. Non-response analysis A non-response analysis was performed, using the Chi-square, Student’s T-Test and Mann–Whitney U test, to determine whether there were differences in characteristics (gender, age and time interval between surgery and sending out the questionnaire) of the participants who did or did not respond to the SPOC-NL questionnaire. We used IBM SPSS Statistics for Windows (Version 20.0, Armonk, NY: IBM Corp.) and G-String_IV software (version 6.2.1.2) for all statistical analyses. A p-value of
