Qual Life Res (2015) 24:417–425 DOI 10.1007/s11136-014-0779-4
Quality of life after pulmonary embolism: first cross-cultural evaluation of the pulmonary embolism quality-of-life (PEmbQoL) questionnaire in a Norwegian cohort Mazdak Tavoly • Lars-Petter Jelsness-Jørgensen Hilde Skuterud Wik • Christina Roaldsnes • Per Morten Sandset • Waleed Ghanima
•
Accepted: 5 August 2014 / Published online: 14 August 2014 Ó Springer International Publishing Switzerland 2014
Abstract Purpose The aim of the current study was to translate and test the psychometrical properties of the disease-specific pulmonary embolism quality-of-life questionnaire (PEmbQoL). Methods Patients with a prior history of pulmonary embolism (PE) were identified from the thrombosis registry at Østfold Hospital Trust, Fredrikstad, Norway. All eligible patients were asked to complete the generic EuroQol 5-dimension (EQ-5D) QoL questionnaire as well as the disease-specific PEmb-QoL at baseline and after 2 weeks. Construct validity was tested using principal component factor analysis. Criterion validity was tested using Spearman’s correlation coefficients (rho) between EQ-5D and PEmb-QoL. Internal consistency reliability was calculated using Cronbach’s alpha coefficient, while test–retest reliability was calculated using the intra-class correlation coefficients (ICC).
M. Tavoly L.-P. Jelsness-Jørgensen C. Roaldsnes W. Ghanima Østfold Hospital Trust, Fredrikstad, Norway M. Tavoly (&) Department of Medicine, Sahlgrenska University Hospital, Per Dubbsgatan 15, 413 45 Gothenburg, Sweden e-mail: [email protected] M. Tavoly P. M. Sandset W. Ghanima Institute of Clinical Medicine, University of Oslo, Oslo, Norway L.-P. Jelsness-Jørgensen Østfold University College, Fredrikstad, Norway H. S. Wik P. M. Sandset Department of Haematology, Oslo University Hospital, Oslo, Norway
Results A total of 213 participants had complete datasets and were included in further analyses. Factor analysis with varimax rotation yielded six factors explaining 71 % of the cumulative variance. Cronbach’s alpha coefficient was found to be 0.94, indicating a very good intercorrelation of items. Of the 213 participants, 145 (68 %) completed the questionnaire a second time. The ICC ranged from 0.75 to 0.86, indicating good test–retest reliability. All factors were found significant with p values \0.001. The criterion validity of the PEmb-QoL was confirmed through good correlation with other similar health-related quality-of-life constructs in the EQ-5D. Conclusions Findings of the current study indicate that Norwegian version of the PEmb-QoL is both valid and reliable, thus representing an important supplement in subjective outcomes measurement among patients sustaining PE. Keywords PEmb-QoL HRQoL Pulmonary embolism Psychometric evaluation Disease-specific questionnaires
Introduction Pulmonary embolism (PE) is a common medical emergency resulting from the obstruction of the pulmonary vasculature by a clot most often originating in deep veins of the lower extremities or the pelvis [1]. The short-term outcomes after PE are well defined. These outcomes range from transient dyspnea and chest pain to hemodynamic instability and death [2, 3]. The long-term outcomes include persistent dyspnea and development of chronic thromboembolic pulmonary hypertension [4]. Adding the risk of bleeding associated with the use of anticoagulation, these outcomes may have an impact on various aspects related to health-related quality of life (HRQoL).
123
418
While the early clinical outcomes after PE have been extensively studied, there has been little focus on long-term outcomes including psychological functioning. HRQoL after PE is a novel research field, which has recently gained increasing interest [5–7]. HRQoL is measured either by generic and/or disease-specific questionnaires, where a combined generic and disease-specific approach is known to increase sensitivity [8]. Disease-specific HRQoL questionnaires are often regarded as more sensitive than generic questionnaires, at least for outcomes that might be clinically relevant in specific patient populations [9]. Until recently, no disease-specific questionnaire was available for PE. The pulmonary embolism quality-of-life (PEmbQoL) questionnaire was developed to close this gap [10]. It has recently been used and validated in a Dutch study [7, 11], and to our knowledge, it still remains the only available disease-specific questionnaire in this particular group of patients [11]. The aim of the present study was to translate the PEmbQoL questionnaire into Norwegian and to validate it in a cohort of patients with a history of PE.
Materials and methods Patients Patients who were diagnosed and treated for PE at the Østfold Hospital Trust, Fredrikstad, Norway, between January 2002 and December 2011 were identified from the thrombosis registry of the Østfold Hospital and by searching hospital databases for ICD-10 codes of PE (ICD-10 I26.0 and I26.9). All living patients who had an objectively confirmed diagnosis of PE by ventilation–perfusion scan or computed tomography pulmonary angiogram (CTPA) scan were invited to participate. Patients were excluded if they were \18 or [90 years, deemed incapable of complying with the study procedure, such as language barriers, cognitive problems, or dementia, and if they were geographically unavailable. Moreover, patients living in nursing homes and those who had a major psychiatric diagnosis, including schizophrenia and major depression, were excluded. Five patients from the original cohort were erroneously not invited to participate in the study. The study was performed according to the principles founded in the revised Declaration of Helsinki, and written informed consent was obtained from each individual before entering the study. The study was approved by the Regional Committee for Medical and Health Research Ethics, Norway (REK 2011/2557). Questionnaires The pulmonary embolism quality-of-life (PEmb-QoL) questionnaire and the EuroQol 5-dimension, 3-level (EQ-
123
Qual Life Res (2015) 24:417–425
5D-3L) forms were sent to the patients either by e-mail or by post. Patients were asked to complete the questionnaires at home and return them at scheduled study visits. Incomplete forms were completed during the visit at the hospital. In order to determine test–retest reliability, patients were asked to complete the forms a second time. These were sent to the patients 2 weeks after the physical evaluation. Incomplete forms were completed by telephone interview. The pulmonary embolism quality-of-life (PEmb-QoL) questionnaire The PEmb-QoL questionnaire is a disease-specific QoL questionnaire [10, 11], which consists of nine questions with a total of 38 single items. A hypothesized dimensional structure was created based on the contents of the items consisting of frequency of complaints (PEmb question one, consisting of eight items), activities of daily life limitations (ADL) (PEmb question four, consisting of 13 items), workrelated problems (PEmb question five, consisting of four items), social limitations (PEmb question six, consisting of one item), intensity of complaints (PEmb question seven and eight, consisting of two items), and emotional complaints (PEmb question nine, consisting of ten items). Two items (i.e., items two and three) are descriptive. In the dimensions frequency of complaints (question one), ADL limitations (question four), work-related problems (question five), and emotional complaints (question nine) items are reversely scored. The mean dimensional scores are calculated by dividing the respondents score in that particular dimension by the number of the items in the dimension. Consequently, the minimum and maximum score of, for example, the frequency of complaints dimension will be ranging from one (lowest possible score) to five (highest possible score). The questionnaire was found both valid and reliable in its original validation [11] and has been translated into English according to the standard guidelines [12]. EuroQoL-5 dimension-3 level (EQ-5D-3L) questionnaire The EQ-5D-3L is a generic HRQoL questionnaire that is widely used [13, 14]. It contains one descriptive system and the EQ visual analogue scale (EQ-VAS). The descriptive system has 5 dimensions: mobility, self-care, usual activities, pain/discomfort, and anxiety/depression. Each dimension has 3 levels: no problems, some problems, and extreme problems. The EQ-VAS is self-rated health on a vertical visual analogue scale (0–100) where the end points are labelled ‘‘best imaginable health state’’ and ‘‘worst imaginable health state.’’ The questionnaire has been found valid and reliable in several languages as well as in Norwegian.
Qual Life Res (2015) 24:417–425
419
Translation procedure In order to develop a Norwegian version of the PEmb-QoL questionnaire, a forward–backward translation procedure from English into Norwegian was performed according to the recommendations in the literature [12]. Two independent translations into Norwegian were performed by two professional bilingual translators, one with a medical background and both having Norwegian as their mother tongue. Furthermore, four observers synthesized one translation by merging the two versions. The questionnaire subsequently underwent a backward translation to English by a translator with English as her mother tongue. Finally, three independent individuals evaluated the questionnaire by comparing the English and Norwegian versions with regard to semantic, idiomatic, experiential, and conceptual equivalence. Following these procedures, a final version was approved and tested.
Fig. 1 Flowchart of the patients
Statistical analysis
Results
Means and standard deviations were used to express normally distributed variables, while medians and interquartile range (IQR) were used to express non-normally distributed variables. Construct validity of the PEmb-QoL was tested by factor analysis using principal component analysis with varimax rotation. A criterion cutoff at eigenvalues [1 was used, and factorial loadings [0.40 were considered high. Criterion validity was determined by correlating the dimensions of the PEmb-QoL with the generic EQ-5D-3L. Inter-dimension correlations and criterion validity were calculated using Spearman’s rho correlation coefficient. Internal consistency reliability was calculated with Cronbach’s alpha [15] and was considered adequate at the 0.7 level or above [16]. Test–retest reliability was analyzed using intra-class correlation coefficients (ICC) between the two different measurements at different time points. Missing values were treated in accordance with the recommendations in the literature; if data for half of the items or fewer within a dimension were missing, they were replaced by the mean value of the respondent’s completed items in the same dimension [17]. Cases with more than 50 % of items missing within a dimension were excluded from the analyses. Potential multicollinearity between PEmb-QoL dimensions was determined by calculating variance inflation factor (VIF) and tolerance. A VIF value [5 and a tolerance value \0.20 were regarded indicative of multicollinearity according to the published recommendations [18]. All tests were two-sided with a 5 % significance level. Statistical analysis was performed using the Statistical Package for the Social Sciences (SPSS) version 20 (SPSS INC; Chicago, IL).
A total of 836 patients were identified and assessed for eligibility in this study. A detailed description of the study cohort is given in Fig. 1. Of the 406 eligible patients, 218 agreed to participate and completed the questionnaires at a median time of 3.6 years (IQR 1.8–6.5) after the diagnosis of PE. Five questionnaires had missing values exceeding 50 % and were consequently omitted from the analysis. In seven questionnaires, there were less than 50 % of items missing and these were consequently calculated. In total, 213 patients had evaluable HRQoL data and were included in further analyses. Sociodemographic and clinical characteristics of the patient cohort are summarized in Table 1. Using the original dimensional structure reported by Klok et al. [11], median scores of the six PEmb dimensions were 1.6 (IQR 1.1–2.4, max 5 points) for the dimension frequency of complaints, 1.3 (IQR 1.0–1.8, max 3 points) for the limitations of activities in daily living, 1.3 (IQR 1.0–2.0, max 2 points) for work-related problems, 1.0 (IQR 1.0–2.0, max 5 points) for social limitations, 2.5 (IQR 1.5–3.0, max 6 points) for intensity of complaints, and 1.5 (IQR 1.2–2.2, max 6 points) for emotional complaints. Psychometrical characteristics Validity Factor analysis (principal component method) with varimax rotation yielded six factors explaining 71 % of the cumulative variance. Factorial loadings are presented as a pattern matrix in Table 2 and scree plot in Fig. 2. After organizing the items according to highest factorial loadings ([0.40), factor one loaded mostly for items concerning activities of daily life.
123
420
Qual Life Res (2015) 24:417–425
Table 1 Sociodemographic and clinical characteristics Variable
N = 213
Female, n (%)
95 (44.6)
Age in years, mean (SD)
56.7 (15.1)
Age at inclusion, mean (SD)
60.1 (15.3)
Years since diagnosis, median (IQR)
3.6 (1.8–6.5)
Sick leave, n (%)
12 (5.8)
Disability pension, n (%)
35 (16.4)
Retired, n (%)
85 (39.9)
Diagnosis, n (%) PE
150 (70.4)
PE ? DVT
63 (29.6)
PE localization, n (%) Unilateral
57 (26.8)
Bilateral
142 (66.7)
Not specified
14 (6.5)
Risk factors, n (%) Prior DVT
29 (13.6)
Prior PE
9 (4.2)
Hereditary disposition
40 (18.8)
Contraceptive pills/HRT
25 (11.7)
Immobilization
24.4 (24.4)
Cancer/chemotherapy last 6 months
16 (7.5)
BMI [30
5 (2.3)
For the calculation of criterion validity, Spearman’s correlation coefficient was used to compare the extracted factors in PEmb-QoL to the EQ-5D-3L dimensions. Factors one, three, five, and six had strong correlations to usual activities in EQ-5D-3L, whereas factor two (emotional complaints) correlated with anxiety and factor four (pain) correlated with pain/discomfort (Table 3). Floor and ceiling effects were calculated by estimating the proportion of patients scoring either the lowest or highest possible score in each of the factors. No ceiling effects were observed, but in factors two to six, floor effects above the recommended limit of 20 % were observed (Table 4) [19]. A sum score of the PEmb-QoL was calculated by summarizing each patient’s dimensional crude score divided by the number of dimensions reported by Klok et al. [11]. A possible scale score thus ranged from 6 to 27; higher score indicated a worse HRQoL. The median sum score in the current study was found to be 9.5 (IQR 7.5–13.0), range 6.2–24.3. Test for multicollinearity revealed satisfactory values for both VIF and tolerance and within the recommended limits of \5 and [0.20, respectively. The range of VIF and tolerance was 1.9–3.2 and 0.32–0.54, respectively.
Treatment, n (%) LMWH only
22 (10.3)
LMWH ? warfarin
182 (85.4)
Additional systemic thrombolysis
3 (1.4)
Treatment complications, n (%) No hemorrhage Hemorrhage
207 (97.2) 2 (0.9)
SD standard deviation, IQR interquartile range, PE pulmonary embolism, DVT deep venous thrombosis, HRT hormone replacement therapy, LMWH low-molecular-weight heparin
Factor two loaded predominantly for emotional complaints. Factor three appeared to load mainly for work-related problems, as did factor six, and difficulties performing regular activities. Factor four loaded for remnant discomfort or pain in the chest and/or back. Factor five was assessed as being associated with pain and fear associated with the sensation that the disease was not fully healed. Factor six consisted merely of item 4a, and a total of 61.5 % of the respondents (n = 131) reported that they did not work. All correlations between the items and their hypothesized scales were above 0.40 (item internal consistency), and the correlation between the items and their hypothesized scales was statistically higher ([2 standard errors) than the correlation between the items and the other scales (item discriminant ability).
123
Reliability The Cronbach’s alpha coefficient measuring internal consistency reliability was found to be 0.94 prior to calculating missing items (n = 7). The alpha increased further to 0.95 after correction of these items, indicating a very good intercorrelation of items. Cronbach’s alpha values are shown in Table 5. All participants (n = 213) were invited to complete the questionnaires a second time. Of all invited patients, 145 (68 %) completed and returned the second questionnaire. There were no differences between responders and non-responders related to gender, but non-responders were significantly younger and had shorter disease duration than responders (p \ 0.05). The ICC is used to quantify the extent in which the results from measurements at two different time points resemble each other. A score of 0 and 1 in ICC values represents no and high reliability, respectively. The factorial solution of the Norwegian PEmb-QoL was used to evaluate test–retest reliability between the two time points. The ICC in the current study varied from 0.75 for factor five to 0.86 for factor one, indicating good test–retest reliability. All factors were found significant with p values \0.001. The details of ICCs are presented in Table 5.
Qual Life Res (2015) 24:417–425
421
Table 2 Factorial structure of the Norwegian version of the PEmbQoL questionnaire PEmb items
Table 2 continued PEmb items
Factorial loadings F1
F2
F3
F4
F5
Factor 1: Limitations in activity I
Factorial loadings F1
F2
9h. Were you afraid to exert yourself?
0.49
0.53
0.49
0.60
F3
F4
F5
F6
F6
1h. Difficulty in breathing or breathlessness
0.44
0.40
9i. Did you feel limited in taking a trip?
4b. Daily activities at home
0.66
0.53
9j. Were you afraid of being alone?
4c. Social activities
0.56
4e. Moderate activities
0.73
4f. Lifting and carrying groceries 4g. Climbing several flights of stairs
0.71
4h. Climbing one flight of stairs
0.73
5b. Accomplished less than you would like
0.73
4i. Bending, kneeling, and squatting 4j. Walking more than half a mile
0.62
5c. Were limited in the kind of work or other activities
0.76
0.76
4k. Walking a couple of hundred yards
0.83
5d. Had difficulty performing the work or other activities
4l. Walking about one hundred yards 4m. Washing or dressing yourself
0.84
1a. Pain behind or between the shoulder blades
0.78
0.54
1b. Pain on or in the chest
0.75
8. How much breathlessness have you experienced in the past 4 weeks?
0.50
0.62
Factor 3: Limitations in activity II
0.66
0.42
4d. Vigorous activities
0.43
0.56
0.48
0.60
0.43
5a. Cut down the amount of time you spent on work or other activities
0.42
0.75
Factor 4: Pain
1c. Pain in the back 0.42
Factor 2: Emotional complaints 1f. Burning sensation in the lungs
0.46
6. During the past 4 weeks, to what extent have your lung symptoms interfered with your normal social activities with family, friends, neighbors, or groups?
0.57
9a. Where you worried about having another pulmonary embolism?
0.52
9b. Did you feel irritable? 9d. Did you become emotional more readily?
0.71 0.84
9e. Did it bother you that you became emotional more quickly?
0.88
9f. Were you depressed or in low spirits?
0.82
9g. Did you feel that you were a burden to your family and friends?
0.74
0.43
1d. Sensation of pressure 7. How much pain around your shoulder blades/pain in your chest have you experienced during the past 4 weeks?
0.71 0.42
0.46 0.79
Factor 5: Treatment
0.41
1e. Feeling that there is still ‘‘something there’’
0.57
1g. Nagging feeling in the lungs
0.57
9c. Would you have been worried if you had to stop taking anticoagulant medication?
0.65
Factor 6: Professional work 4a. Daily activities at work
0.72
Principle component analysis performed using varimax rotation and eigenvalues [1, coefficients \0.40 were omitted from the Table, highest factorial loadings are in bold face PEmb-QoL pulmonary embolism quality of life
Discussion The current study was performed to translate the PEmbQoL questionnaire into Norwegian and to evaluate its
123
422
Qual Life Res (2015) 24:417–425
Fig. 2 Scree plot of the Norwegian version of the pulmonary embolism qualityof-life questionnaire (PembQoL)
Table 3 External validity of the Norwegian version of the PEmb-QoL questionnaire, using Spearman’s correlation analysis with the EQ-5D-3L questionnaire EQ-5D-3L factors
F1
F2
F3
F4
F5
F6
Mobility
0.46±
0.44±
0.45±
0.29±
0.18}
Self-care
0.33
±
±
0.25
0.33
±
}
0.15
?
0.07
-0.22*
Usual activities
0.63–
0.56±
0.62–
0.49±
0.35–
20.29–
Pain/discomfort
0.54±
0.52±
0.44±
0.64–
0.27±
-0.23*
±
–
±
±
0.31±
-0.21*
-0.51±
-0.35±
-0.28±
Anxiety/depression
0.31
-0.67±
EQ-VAS
0.59
0.30
-0.55±
-0.61±
0.33
-0.27±
Factors displaying the highest correlations are in bold face PEmb-QoL pulmonary embolism quality of life Significance levels:
±
\0.001, * \ 0.01, } \ 0.05,
?
nonsignificant
psychometrical properties. To our knowledge, this is the first cross-cultural evaluation [10, 11]. Our findings indicate that the Norwegian version is both valid and reliable, thus representing an important supplement for measuring subjective outcomes in patients who have sustained PE. Klok et al. [11] established a hypothesized dimensional structure, consisting of six dimensions, which concur with a majority of questions in the questionnaire (except from questions two and three). Basing the construct of a questionnaire merely on a postulated structure may be debateable, since no consideration is made about the underlying structure. Even though it might be assumed that the items are highly correlated, such a structure can only be confirmed and tested by performing a factor analysis. Formal factor analysis consequently plays a major role in construct validation [17]. Deciding which factors to retain is crucial.
123
Table 4 Median values and floor effects of factors extracted in the Norwegian PEmb-QoL PEmb factors
Median
Interquartile range
Floor effects (%)
F1
1.5
1.2–2.2
16
F2
1.4
1.1–2.0
20.2
F3
1.4
1.0–2.2
26.3
F4
1.6
1.0–2.4
33.8
F5
1.7
1.7–2.7
28.6
F6
0.0
0.0–1.0
61.5
PEmb-QoL pulmonary embolism quality of life, F1–F6 extracted factors 1–6 based on principle component analysis
Over- or under-extraction may have deleterious effects on the results [20, 21]. It is debated which method to be used in the factor-retaining process [22], but generally it is
Qual Life Res (2015) 24:417–425
423
Table 5 Test–retest reliability of the Norwegian version of the PEmb-QoL questionnaire (n = 146) Cronbach’s alpha
ICC?
p value
1.4 (0.58)
0.93
0.86
\0.001
1.3 (0.75)
0.88
0.79
\0.001
1.4 (0.49) 1.6 (0.96)
1.4 (0.46) 1.4 (0.90)
0.89 0.90
0.81 0.83
\0.001 \0.001
F5
1.7 (1.07)
1.7 (0.97)
0.86
0.75
\0.001
F6
0.0 (0.60)
0.0 (0.61)
0.88
0.78
\0.001
PEmb N factors
Median (SD) Baseline
Retest
F1
1.5 (0.60)
F2
1.4 (0.79)
F3 F4
PEmb-QoL pulmonary embolism quality of life, ICC intraclass correlation coefficient, ? two-way mixed model—single measures, SD standard deviation
recommended that eigenvalue [1 should be accompanied by another method, e.g., scree plot [20]. However, retaining factors based on only eigenvalue is still an extensively applied approach [17, 23]. In the original article by Klok et al. [11], it is not stated which method was used. The principal component analysis in the current study yielded six factors explaining 71 % of the cumulative variance. The varimax rotation clustered the items fairly consistent with the original dimensional structure, except for factors five and six. Factor five loaded for items 1e, 1g, and 9c, respectively. The first two of these items originate from the frequency of complaints dimension and the last one from the emotional complaints dimension. A logical explanation may be that these questions are interpreted as if the disease has not been fully recovered. Factor six had its highest loading in item 4a, which originate from the ADL limitations dimension, and it could have been expected to load in factor three, clustering items about work-related problems. The reason for this is not clear, but a potential explanation may be that 4a is the only item addressing work per se, which of course not necessarily applies to a large amount of the patients in this population. The latter is underscored by the fact that 61.5 % of patients included in this study reported that they did not work. Moreover, it is well known from studies performed in different patient populations [24] that the number of factor in the same questionnaire may differ across various studies in different countries. Furthermore, discrepancies may be found related to which items that relates to the various factors. The same phenomenon was seen in this study, reproducing the same amount of factors as in the original validation, but with a different item loading. It might be speculated whether observed differences in factorial structure and score may be related to potential differences in clinical and sociodemographic variables between the current study and the study by Klok et al. [11]. Compared to the study by Klok et al. [11], our study had more patients included (n = 213 vs. n = 90), less patients had active
malignancy and less patients were obese. Patients in our cohort were, however, slightly older at the time of inclusion compared to patients in the study by Klok et al. [11] (60 vs. 56 years). Even though not directly comparable, we cannot exclude that these factors may potentially have contributed to the observed differences. There are obvious problems related to such findings. In particular, it may limit the possibility of comparison across studies and boundaries. A possible solution to circumvent this problem may be to calculate and compare a PEmb-QoL sum score. The sum score was, however, not reported in the original validation [11]. It may also be debateable to merely report the sum score on the basis that HRQoL is generally viewed as a multidimensional construct [17]. Sample size is much debated in exploratory factor analysis, and there is no unanimous understanding or presented algorithms for adequate sample size [25, 26]. Strict rules about sample size have mostly disappeared [20, 27]. Traditionally, it has been considered that ‘‘the more the better’’ [25]. Several guidelines mention the subject to item ratio as a common method to be used [28]. Our subject to item ratio was 5.5:1, which is a ratio that is advocated by some and commonly used in several other studies [23, 29]. Internal consistency reliability showed excellent results, way over the recommended limit [24]. However, an excellent Cronbach’s alpha may be a marker for redundancy [21]. In other words, one or more items are basically asking the same question or at least the subject is interpreting the questions in the same way. Very high Cronbach’s alpha can also be found in scales with large number of items, because it is dependent upon the number of items in a scale [27]. In order to investigate whether the PEmbQoL dimensions displayed ambiguous multicollinearity, both variance inflation factor (VIF) and tolerance were calculated. No questionable values were displayed indicating that no problematic multicollinearity was found. There is no general consensus as to how long the between-assessment time gap should be when measuring test–retest reliability. It is generally agreed, however, that a too short test–retest period can result in patients’ recalling their answers from baseline, and a too long period may allow true change in the patients’ clinical condition [17]. In our study, questionnaires were sent to patients in order to calculate test–retest reliability after 2 weeks, meaning that patients completed them at a minimum of 14 days after our initial assessment. In comparison with, e.g., cancer where one would anticipate HRQoL to vary over time, the risk of such variations in the present population was viewed to be low. Furthermore, a retest interval of minimum 14 days was judged as adequate to avoid a recall bias. Findings indicated that the test–retest reliability was good with ICC values consistently over 0.70, which is recommended by some guidelines [27, 30]. Thus, the questionnaire seems to
123
424
have adequate reproducibility and consequently suitable for longitudinal comparisons. Criterion validity was confirmed by rather good correlations between the PEmb-QoL factors and the EQ-5D-3L dimensions. Factors one, three, five, and six (related to ADL and work/regular activity) correlated with EQ-5D3L’s dimension for usual activities. Factor four (pain) correlated significantly with pain in EQ-5D-3L. Finally, factor two (emotional complaints) correlated significantly with anxiety in EQ-5D-3L. The main limitations of the study are the retrospective patients’ identification and the incomplete participation of the original cohort. One might speculate that higher percentage of patients with both reduced health-related quality of life and increased symptom burden agreed to participate. However, given the fact that a marked floor effect was observed in five out of six factors (indicating minor HRQoL problems), this seems unlikely. Problematic floor effects were also reported by Klok et al. [11], and the most plausible explanation seems to be related to the fact that patients in the current study were recruited from a hospital registry, with a median time since diagnosis being 3.6 years. Consequently, a majority of these patients may no longer have troubling symptoms. Furthermore, patients did not rate their symptoms at the two measurement times, which would have been valuable in order to calculate responsiveness (sensitivity to change). Indeed, responsiveness is an indicator of the ability of a questionnaire to detect changes over time and should consequently be evaluated in longitudinal studies [17]. Given the rather long time since the diagnosis of PE, real changes in patients’ health condition may have occurred at an earlier stage, and it is difficult to know whether a symptom score would in fact be related to the PE diagnosis. Moreover, the potential improved level of function over time may have resulted in a response shift among these patients [31]. On the other hand, since this study had a fairly large sample size and inclusion criteria were broad, we judge this cohort to be a representative sample of prior PE patients. We recommend, however, that floor effects and sensitivity to change should be further investigated in future longitudinal follow-up studies of patients with PE. In the current study, EQ-5D-3L was the chosen generic HRQoL instrument. This choice was based on the brevity of the EQ-5D-3L and consequently the potential positive influence this may have on patients’ compliance and completeness of scores. In retrospect, however, the Short-Form 36 (SF-36) might have been more a better choice because of its comprehensiveness for measuring generic HRQoL. There are also some limitations attached to the translation procedure of the PEmb-QoL. Indeed, response choice ratings could have been used in order to test the conformity of the response choices to their hypothesized ordinality. Besides, even though no
123
Qual Life Res (2015) 24:417–425
differences were seen related to response rates when distributing questionnaires either by post or e-mail, questions may be raised related to potential influence on validation. In our view, this seems to be a minor problem, given the fact that questionnaires (regardless of receiving them by post or e-mail) were completed at home. Consequently, having some patients filling them out in the hospital and some at home would have constituted a greater problem. In conclusion, the results from the psychometrical testing of the PEmb-QoL in this Norwegian cohort are promising and indicate overall good reliability, validity, and reproducibility; however, further studies are warranted to evaluate the questionnaire with regard to sensitivity to change (responsiveness) and potential response shifts [32]. Currently, there are no longitudinal follow-up studies from the time of diagnoses of PE using disease-specific measures. Such studies are warranted as well as the need to test the PEmb-QoL in studies aiming to evaluate various treatment effects and interventions.
References 1. Kearon, C. (2003). Natural history of venous thromboembolism. Circulation, 107(23 Suppl 1), I22–I30. 2. Carson, J. L., Kelley, M. A., Duff, A., Weg, J. G., Fulkerson, W. J., Palevsky, H. I., et al. (1992). The clinical course of pulmonary embolism. New England Journal of Medicine, 326(19), 1240–1245. 3. Goldhaber, S. Z., Visani, L., & De Rosa, M. (1999). Acute pulmonary embolism: Clinical outcomes in the International Cooperative Pulmonary Embolism Registry (ICOPER). The Lancet, 353(9162), 1386–1389. 4. Pengo, V., Lensing, A. W., Prins, M. H., Marchiori, A., Davidson, B. L., Tiozzo, F., et al. (2004). Incidence of chronic thromboembolic pulmonary hypertension after pulmonary embolism. New England Journal of Medicine, 350(22), 2257–2264. 5. Klok, F. A., Tijmensen, J. E., Haeck, M. L., van Kralingen, K. W., & Huisman, M. V. (2008). Persistent dyspnea complaints at long-term follow-up after an episode of acute pulmonary embolism: Results of a questionnaire. European Journal of Internal Medicine, 19(8), 625–629. 6. Klok, F. A., van Kralingen, K. W., van Dijk, A. P., Heyning, F. H., Vliegen, H. W., Kaptein, A. A., et al. (2010). Quality of life in long-term survivors of acute pulmonary embolism. Chest, 138(6), 1432–1440. 7. van Es, J., den Exter, P. L., Kaptein, A. A., Andela, C. D., Erkens, P. M., Klok, F. A., et al. (2013). Quality of life after pulmonary embolism as assessed with SF-36 and PEmb-QoL. Thrombosis Research, 132(5), 500–505. 8. Hawker, G., Melfi, C., Paul, J., Green, R., & Bombardier, C. (1995). Comparison of a generic (SF-36) and a disease specific (WOMAC) (Western Ontario and McMaster Universities Osteoarthritis Index) instrument in the measurement of outcomes after knee replacement surgery. Journal of Rheumatology, 22(6), 1193–1196. 9. Salaffi, F., Carotti, M., & Grassi, W. (2005). Health-related quality of life in patients with hip or knee osteoarthritis: Comparison of generic and disease-specific instruments. Clinical Rheumatology, 24(1), 29–37.
Qual Life Res (2015) 24:417–425 10. Cohn, D. M., Nelis, E. A., Busweiler, L. A., Kaptein, A. A., & Middeldorp, S. (2009). Quality of life after pulmonary embolism: the development of the PEmb-QoL questionnaire. Journal of Thrombosis and Haemostasis, 7(6), 1044–1046. 11. Klok, F. A., Cohn, D. M., Middeldorp, S., Scharloo, M., Buller, H. R., van Kralingen, K. W., et al. (2010). Quality of life after pulmonary embolism: Validation of the PEmb-QoL questionnaire. Journal of Thrombosis and Haemostasis, 8(3), 523–532. 12. Beaton, D. E., Bombardier, C., Guillemin, F., & Ferraz, M. B. (2000). Guidelines for the process of cross-cultural adaptation of self-report measures. Spine (Phila Pa 1976), 25(24), 3186–3191. 13. EuroQol, G. (1990). EuroQol—A new facility for the measurement of health-related quality of life. Health Policy, 16(3), 199–208. 14. Brooks, R. (1996). EuroQol: the current state of play. Health Policy, 37(1), 53–72. 15. Cronbach, L. J. (1951). Coefficient alpha and the internal structure of tests. Psychometrika, 16(3), 297–334. 16. DeVellis, R. F. (1991). Scale development: Theory and applications (1st ed.). Newbury Park, CA: Sage. 17. Fayers, P. M., & Machin, D. (2007). Quality of life : The assessment, analysis and interpretation of patient-reported outcomes (2nd ed.). Chichester: Wiley. 18. Hocking, R. R. (2003). Methods and applications of linear models: Regression and the analysis of variance. Hoboken, New Jersey: Wiley. 19. McHorney, C. A., Ware, J. E, Jr, Lu, J. F., & Sherbourne, C. D. (1994). The MOS 36-item Short-Form Health Survey (SF-36): III. Tests of data quality, scaling assumptions, and reliability across diverse patient groups. Medical Care, 32(1), 40–66. 20. Costello, A. B., & Osborne, J. W. (2005). Best practices in exploratory factor analysis: Four recommendations for getting the most from your analysis. Practical Assessment, Research & Evaluation, 10, 173–178. 21. Pesudovs, K., Burr, J. M., Harley, C., & Elliott, D. B. (2007). The development, assessment, and selection of questionnaires. Optometry and Vision Science, 84(8), 663–674.
425 22. Zwick, W. R., & Velicer, W. F. (1986). Comparison of five rules for determining the number of components to retain. Psychological Bulletin, 99(3), 432. 23. De Vet, H. C. W., Ade`r, H. J., Terwee, C. B., & Pouwer, F. (2005). Are factor analytical techniques used appropriately in the validation of health status questionnaires? A systematic review on the quality of factor analysis of the SF-36. Quality of Life Research, 14(5), 1203–1218. 24. Aaronson, N., Alonso, J., Burnam, A., Lohr, K. N., Patrick, D. L., Perrin, E., et al. (2002). Assessing health status and quality-of-life instruments: Attributes and review criteria. Quality of Life Research, 11(3), 193–205. 25. Floyd, F. J., & Widaman, K. F. (1995). Factor analysis in the development and refinement of clinical assessment instruments. Psychological Assessment, 7(3), 286–299. 26. Guadagnoli, E., & Velicer, W. F. (1988). Relation of sample size to the stability of component patterns. Psychological Bulletin, 103(2), 265–275. 27. Terwee, C. B., Bot, S. D., de Boer, M. R., van der Windt, D. A., Knol, D. L., Dekker, J., et al. (2007). Quality criteria were proposed for measurement properties of health status questionnaires. Journal of Clinical Epidemiology, 60(1), 34–42. 28. Streiner, D. L. (1994). Figuring out factors: The use and misuse of factor analysis. Canadian Journal of Psychiatry, 39(3), 135–140. 29. Kline, P. (1993). The handbook of psychological testing (2nd ed.). London: Routledge. 30. Nunnally, J. C., & Bernstein, I. H. (1994). Psychometric theory (3rd ed.). New York: McGraw-Hill. 31. Calvert, M., Blazeby, J., Altman, D. G., Revicki, D. A., Moher, D., Brundage, M. D., et al. (2013). Reporting of patient-reported outcomes in randomized trials: The CONSORT PRO extension. JAMA, 309(8), 814–822. 32. Schwartz, C. E., Andresen, E. M., Nosek, M. A., Krahn, G. L., & Measurement, R. E. P. o. H. S. (2007). Response shift theory: Important implications for measuring quality of life in people with disability. Archives of Physical Medicine and Rehabilitation, 88(4), 529–536.
123
Quality of life after pulmonary embolism: first cross-cultural evaluation of the pulmonary embolism quality-of-life (PEmbQoL) questionnaire in a Norwegian cohort Mazdak Tavoly • Lars-Petter Jelsness-Jørgensen Hilde Skuterud Wik • Christina Roaldsnes • Per Morten Sandset • Waleed Ghanima
•
Accepted: 5 August 2014 / Published online: 14 August 2014 Ó Springer International Publishing Switzerland 2014
Abstract Purpose The aim of the current study was to translate and test the psychometrical properties of the disease-specific pulmonary embolism quality-of-life questionnaire (PEmbQoL). Methods Patients with a prior history of pulmonary embolism (PE) were identified from the thrombosis registry at Østfold Hospital Trust, Fredrikstad, Norway. All eligible patients were asked to complete the generic EuroQol 5-dimension (EQ-5D) QoL questionnaire as well as the disease-specific PEmb-QoL at baseline and after 2 weeks. Construct validity was tested using principal component factor analysis. Criterion validity was tested using Spearman’s correlation coefficients (rho) between EQ-5D and PEmb-QoL. Internal consistency reliability was calculated using Cronbach’s alpha coefficient, while test–retest reliability was calculated using the intra-class correlation coefficients (ICC).
M. Tavoly L.-P. Jelsness-Jørgensen C. Roaldsnes W. Ghanima Østfold Hospital Trust, Fredrikstad, Norway M. Tavoly (&) Department of Medicine, Sahlgrenska University Hospital, Per Dubbsgatan 15, 413 45 Gothenburg, Sweden e-mail: [email protected] M. Tavoly P. M. Sandset W. Ghanima Institute of Clinical Medicine, University of Oslo, Oslo, Norway L.-P. Jelsness-Jørgensen Østfold University College, Fredrikstad, Norway H. S. Wik P. M. Sandset Department of Haematology, Oslo University Hospital, Oslo, Norway
Results A total of 213 participants had complete datasets and were included in further analyses. Factor analysis with varimax rotation yielded six factors explaining 71 % of the cumulative variance. Cronbach’s alpha coefficient was found to be 0.94, indicating a very good intercorrelation of items. Of the 213 participants, 145 (68 %) completed the questionnaire a second time. The ICC ranged from 0.75 to 0.86, indicating good test–retest reliability. All factors were found significant with p values \0.001. The criterion validity of the PEmb-QoL was confirmed through good correlation with other similar health-related quality-of-life constructs in the EQ-5D. Conclusions Findings of the current study indicate that Norwegian version of the PEmb-QoL is both valid and reliable, thus representing an important supplement in subjective outcomes measurement among patients sustaining PE. Keywords PEmb-QoL HRQoL Pulmonary embolism Psychometric evaluation Disease-specific questionnaires
Introduction Pulmonary embolism (PE) is a common medical emergency resulting from the obstruction of the pulmonary vasculature by a clot most often originating in deep veins of the lower extremities or the pelvis [1]. The short-term outcomes after PE are well defined. These outcomes range from transient dyspnea and chest pain to hemodynamic instability and death [2, 3]. The long-term outcomes include persistent dyspnea and development of chronic thromboembolic pulmonary hypertension [4]. Adding the risk of bleeding associated with the use of anticoagulation, these outcomes may have an impact on various aspects related to health-related quality of life (HRQoL).
123
418
While the early clinical outcomes after PE have been extensively studied, there has been little focus on long-term outcomes including psychological functioning. HRQoL after PE is a novel research field, which has recently gained increasing interest [5–7]. HRQoL is measured either by generic and/or disease-specific questionnaires, where a combined generic and disease-specific approach is known to increase sensitivity [8]. Disease-specific HRQoL questionnaires are often regarded as more sensitive than generic questionnaires, at least for outcomes that might be clinically relevant in specific patient populations [9]. Until recently, no disease-specific questionnaire was available for PE. The pulmonary embolism quality-of-life (PEmbQoL) questionnaire was developed to close this gap [10]. It has recently been used and validated in a Dutch study [7, 11], and to our knowledge, it still remains the only available disease-specific questionnaire in this particular group of patients [11]. The aim of the present study was to translate the PEmbQoL questionnaire into Norwegian and to validate it in a cohort of patients with a history of PE.
Materials and methods Patients Patients who were diagnosed and treated for PE at the Østfold Hospital Trust, Fredrikstad, Norway, between January 2002 and December 2011 were identified from the thrombosis registry of the Østfold Hospital and by searching hospital databases for ICD-10 codes of PE (ICD-10 I26.0 and I26.9). All living patients who had an objectively confirmed diagnosis of PE by ventilation–perfusion scan or computed tomography pulmonary angiogram (CTPA) scan were invited to participate. Patients were excluded if they were \18 or [90 years, deemed incapable of complying with the study procedure, such as language barriers, cognitive problems, or dementia, and if they were geographically unavailable. Moreover, patients living in nursing homes and those who had a major psychiatric diagnosis, including schizophrenia and major depression, were excluded. Five patients from the original cohort were erroneously not invited to participate in the study. The study was performed according to the principles founded in the revised Declaration of Helsinki, and written informed consent was obtained from each individual before entering the study. The study was approved by the Regional Committee for Medical and Health Research Ethics, Norway (REK 2011/2557). Questionnaires The pulmonary embolism quality-of-life (PEmb-QoL) questionnaire and the EuroQol 5-dimension, 3-level (EQ-
123
Qual Life Res (2015) 24:417–425
5D-3L) forms were sent to the patients either by e-mail or by post. Patients were asked to complete the questionnaires at home and return them at scheduled study visits. Incomplete forms were completed during the visit at the hospital. In order to determine test–retest reliability, patients were asked to complete the forms a second time. These were sent to the patients 2 weeks after the physical evaluation. Incomplete forms were completed by telephone interview. The pulmonary embolism quality-of-life (PEmb-QoL) questionnaire The PEmb-QoL questionnaire is a disease-specific QoL questionnaire [10, 11], which consists of nine questions with a total of 38 single items. A hypothesized dimensional structure was created based on the contents of the items consisting of frequency of complaints (PEmb question one, consisting of eight items), activities of daily life limitations (ADL) (PEmb question four, consisting of 13 items), workrelated problems (PEmb question five, consisting of four items), social limitations (PEmb question six, consisting of one item), intensity of complaints (PEmb question seven and eight, consisting of two items), and emotional complaints (PEmb question nine, consisting of ten items). Two items (i.e., items two and three) are descriptive. In the dimensions frequency of complaints (question one), ADL limitations (question four), work-related problems (question five), and emotional complaints (question nine) items are reversely scored. The mean dimensional scores are calculated by dividing the respondents score in that particular dimension by the number of the items in the dimension. Consequently, the minimum and maximum score of, for example, the frequency of complaints dimension will be ranging from one (lowest possible score) to five (highest possible score). The questionnaire was found both valid and reliable in its original validation [11] and has been translated into English according to the standard guidelines [12]. EuroQoL-5 dimension-3 level (EQ-5D-3L) questionnaire The EQ-5D-3L is a generic HRQoL questionnaire that is widely used [13, 14]. It contains one descriptive system and the EQ visual analogue scale (EQ-VAS). The descriptive system has 5 dimensions: mobility, self-care, usual activities, pain/discomfort, and anxiety/depression. Each dimension has 3 levels: no problems, some problems, and extreme problems. The EQ-VAS is self-rated health on a vertical visual analogue scale (0–100) where the end points are labelled ‘‘best imaginable health state’’ and ‘‘worst imaginable health state.’’ The questionnaire has been found valid and reliable in several languages as well as in Norwegian.
Qual Life Res (2015) 24:417–425
419
Translation procedure In order to develop a Norwegian version of the PEmb-QoL questionnaire, a forward–backward translation procedure from English into Norwegian was performed according to the recommendations in the literature [12]. Two independent translations into Norwegian were performed by two professional bilingual translators, one with a medical background and both having Norwegian as their mother tongue. Furthermore, four observers synthesized one translation by merging the two versions. The questionnaire subsequently underwent a backward translation to English by a translator with English as her mother tongue. Finally, three independent individuals evaluated the questionnaire by comparing the English and Norwegian versions with regard to semantic, idiomatic, experiential, and conceptual equivalence. Following these procedures, a final version was approved and tested.
Fig. 1 Flowchart of the patients
Statistical analysis
Results
Means and standard deviations were used to express normally distributed variables, while medians and interquartile range (IQR) were used to express non-normally distributed variables. Construct validity of the PEmb-QoL was tested by factor analysis using principal component analysis with varimax rotation. A criterion cutoff at eigenvalues [1 was used, and factorial loadings [0.40 were considered high. Criterion validity was determined by correlating the dimensions of the PEmb-QoL with the generic EQ-5D-3L. Inter-dimension correlations and criterion validity were calculated using Spearman’s rho correlation coefficient. Internal consistency reliability was calculated with Cronbach’s alpha [15] and was considered adequate at the 0.7 level or above [16]. Test–retest reliability was analyzed using intra-class correlation coefficients (ICC) between the two different measurements at different time points. Missing values were treated in accordance with the recommendations in the literature; if data for half of the items or fewer within a dimension were missing, they were replaced by the mean value of the respondent’s completed items in the same dimension [17]. Cases with more than 50 % of items missing within a dimension were excluded from the analyses. Potential multicollinearity between PEmb-QoL dimensions was determined by calculating variance inflation factor (VIF) and tolerance. A VIF value [5 and a tolerance value \0.20 were regarded indicative of multicollinearity according to the published recommendations [18]. All tests were two-sided with a 5 % significance level. Statistical analysis was performed using the Statistical Package for the Social Sciences (SPSS) version 20 (SPSS INC; Chicago, IL).
A total of 836 patients were identified and assessed for eligibility in this study. A detailed description of the study cohort is given in Fig. 1. Of the 406 eligible patients, 218 agreed to participate and completed the questionnaires at a median time of 3.6 years (IQR 1.8–6.5) after the diagnosis of PE. Five questionnaires had missing values exceeding 50 % and were consequently omitted from the analysis. In seven questionnaires, there were less than 50 % of items missing and these were consequently calculated. In total, 213 patients had evaluable HRQoL data and were included in further analyses. Sociodemographic and clinical characteristics of the patient cohort are summarized in Table 1. Using the original dimensional structure reported by Klok et al. [11], median scores of the six PEmb dimensions were 1.6 (IQR 1.1–2.4, max 5 points) for the dimension frequency of complaints, 1.3 (IQR 1.0–1.8, max 3 points) for the limitations of activities in daily living, 1.3 (IQR 1.0–2.0, max 2 points) for work-related problems, 1.0 (IQR 1.0–2.0, max 5 points) for social limitations, 2.5 (IQR 1.5–3.0, max 6 points) for intensity of complaints, and 1.5 (IQR 1.2–2.2, max 6 points) for emotional complaints. Psychometrical characteristics Validity Factor analysis (principal component method) with varimax rotation yielded six factors explaining 71 % of the cumulative variance. Factorial loadings are presented as a pattern matrix in Table 2 and scree plot in Fig. 2. After organizing the items according to highest factorial loadings ([0.40), factor one loaded mostly for items concerning activities of daily life.
123
420
Qual Life Res (2015) 24:417–425
Table 1 Sociodemographic and clinical characteristics Variable
N = 213
Female, n (%)
95 (44.6)
Age in years, mean (SD)
56.7 (15.1)
Age at inclusion, mean (SD)
60.1 (15.3)
Years since diagnosis, median (IQR)
3.6 (1.8–6.5)
Sick leave, n (%)
12 (5.8)
Disability pension, n (%)
35 (16.4)
Retired, n (%)
85 (39.9)
Diagnosis, n (%) PE
150 (70.4)
PE ? DVT
63 (29.6)
PE localization, n (%) Unilateral
57 (26.8)
Bilateral
142 (66.7)
Not specified
14 (6.5)
Risk factors, n (%) Prior DVT
29 (13.6)
Prior PE
9 (4.2)
Hereditary disposition
40 (18.8)
Contraceptive pills/HRT
25 (11.7)
Immobilization
24.4 (24.4)
Cancer/chemotherapy last 6 months
16 (7.5)
BMI [30
5 (2.3)
For the calculation of criterion validity, Spearman’s correlation coefficient was used to compare the extracted factors in PEmb-QoL to the EQ-5D-3L dimensions. Factors one, three, five, and six had strong correlations to usual activities in EQ-5D-3L, whereas factor two (emotional complaints) correlated with anxiety and factor four (pain) correlated with pain/discomfort (Table 3). Floor and ceiling effects were calculated by estimating the proportion of patients scoring either the lowest or highest possible score in each of the factors. No ceiling effects were observed, but in factors two to six, floor effects above the recommended limit of 20 % were observed (Table 4) [19]. A sum score of the PEmb-QoL was calculated by summarizing each patient’s dimensional crude score divided by the number of dimensions reported by Klok et al. [11]. A possible scale score thus ranged from 6 to 27; higher score indicated a worse HRQoL. The median sum score in the current study was found to be 9.5 (IQR 7.5–13.0), range 6.2–24.3. Test for multicollinearity revealed satisfactory values for both VIF and tolerance and within the recommended limits of \5 and [0.20, respectively. The range of VIF and tolerance was 1.9–3.2 and 0.32–0.54, respectively.
Treatment, n (%) LMWH only
22 (10.3)
LMWH ? warfarin
182 (85.4)
Additional systemic thrombolysis
3 (1.4)
Treatment complications, n (%) No hemorrhage Hemorrhage
207 (97.2) 2 (0.9)
SD standard deviation, IQR interquartile range, PE pulmonary embolism, DVT deep venous thrombosis, HRT hormone replacement therapy, LMWH low-molecular-weight heparin
Factor two loaded predominantly for emotional complaints. Factor three appeared to load mainly for work-related problems, as did factor six, and difficulties performing regular activities. Factor four loaded for remnant discomfort or pain in the chest and/or back. Factor five was assessed as being associated with pain and fear associated with the sensation that the disease was not fully healed. Factor six consisted merely of item 4a, and a total of 61.5 % of the respondents (n = 131) reported that they did not work. All correlations between the items and their hypothesized scales were above 0.40 (item internal consistency), and the correlation between the items and their hypothesized scales was statistically higher ([2 standard errors) than the correlation between the items and the other scales (item discriminant ability).
123
Reliability The Cronbach’s alpha coefficient measuring internal consistency reliability was found to be 0.94 prior to calculating missing items (n = 7). The alpha increased further to 0.95 after correction of these items, indicating a very good intercorrelation of items. Cronbach’s alpha values are shown in Table 5. All participants (n = 213) were invited to complete the questionnaires a second time. Of all invited patients, 145 (68 %) completed and returned the second questionnaire. There were no differences between responders and non-responders related to gender, but non-responders were significantly younger and had shorter disease duration than responders (p \ 0.05). The ICC is used to quantify the extent in which the results from measurements at two different time points resemble each other. A score of 0 and 1 in ICC values represents no and high reliability, respectively. The factorial solution of the Norwegian PEmb-QoL was used to evaluate test–retest reliability between the two time points. The ICC in the current study varied from 0.75 for factor five to 0.86 for factor one, indicating good test–retest reliability. All factors were found significant with p values \0.001. The details of ICCs are presented in Table 5.
Qual Life Res (2015) 24:417–425
421
Table 2 Factorial structure of the Norwegian version of the PEmbQoL questionnaire PEmb items
Table 2 continued PEmb items
Factorial loadings F1
F2
F3
F4
F5
Factor 1: Limitations in activity I
Factorial loadings F1
F2
9h. Were you afraid to exert yourself?
0.49
0.53
0.49
0.60
F3
F4
F5
F6
F6
1h. Difficulty in breathing or breathlessness
0.44
0.40
9i. Did you feel limited in taking a trip?
4b. Daily activities at home
0.66
0.53
9j. Were you afraid of being alone?
4c. Social activities
0.56
4e. Moderate activities
0.73
4f. Lifting and carrying groceries 4g. Climbing several flights of stairs
0.71
4h. Climbing one flight of stairs
0.73
5b. Accomplished less than you would like
0.73
4i. Bending, kneeling, and squatting 4j. Walking more than half a mile
0.62
5c. Were limited in the kind of work or other activities
0.76
0.76
4k. Walking a couple of hundred yards
0.83
5d. Had difficulty performing the work or other activities
4l. Walking about one hundred yards 4m. Washing or dressing yourself
0.84
1a. Pain behind or between the shoulder blades
0.78
0.54
1b. Pain on or in the chest
0.75
8. How much breathlessness have you experienced in the past 4 weeks?
0.50
0.62
Factor 3: Limitations in activity II
0.66
0.42
4d. Vigorous activities
0.43
0.56
0.48
0.60
0.43
5a. Cut down the amount of time you spent on work or other activities
0.42
0.75
Factor 4: Pain
1c. Pain in the back 0.42
Factor 2: Emotional complaints 1f. Burning sensation in the lungs
0.46
6. During the past 4 weeks, to what extent have your lung symptoms interfered with your normal social activities with family, friends, neighbors, or groups?
0.57
9a. Where you worried about having another pulmonary embolism?
0.52
9b. Did you feel irritable? 9d. Did you become emotional more readily?
0.71 0.84
9e. Did it bother you that you became emotional more quickly?
0.88
9f. Were you depressed or in low spirits?
0.82
9g. Did you feel that you were a burden to your family and friends?
0.74
0.43
1d. Sensation of pressure 7. How much pain around your shoulder blades/pain in your chest have you experienced during the past 4 weeks?
0.71 0.42
0.46 0.79
Factor 5: Treatment
0.41
1e. Feeling that there is still ‘‘something there’’
0.57
1g. Nagging feeling in the lungs
0.57
9c. Would you have been worried if you had to stop taking anticoagulant medication?
0.65
Factor 6: Professional work 4a. Daily activities at work
0.72
Principle component analysis performed using varimax rotation and eigenvalues [1, coefficients \0.40 were omitted from the Table, highest factorial loadings are in bold face PEmb-QoL pulmonary embolism quality of life
Discussion The current study was performed to translate the PEmbQoL questionnaire into Norwegian and to evaluate its
123
422
Qual Life Res (2015) 24:417–425
Fig. 2 Scree plot of the Norwegian version of the pulmonary embolism qualityof-life questionnaire (PembQoL)
Table 3 External validity of the Norwegian version of the PEmb-QoL questionnaire, using Spearman’s correlation analysis with the EQ-5D-3L questionnaire EQ-5D-3L factors
F1
F2
F3
F4
F5
F6
Mobility
0.46±
0.44±
0.45±
0.29±
0.18}
Self-care
0.33
±
±
0.25
0.33
±
}
0.15
?
0.07
-0.22*
Usual activities
0.63–
0.56±
0.62–
0.49±
0.35–
20.29–
Pain/discomfort
0.54±
0.52±
0.44±
0.64–
0.27±
-0.23*
±
–
±
±
0.31±
-0.21*
-0.51±
-0.35±
-0.28±
Anxiety/depression
0.31
-0.67±
EQ-VAS
0.59
0.30
-0.55±
-0.61±
0.33
-0.27±
Factors displaying the highest correlations are in bold face PEmb-QoL pulmonary embolism quality of life Significance levels:
±
\0.001, * \ 0.01, } \ 0.05,
?
nonsignificant
psychometrical properties. To our knowledge, this is the first cross-cultural evaluation [10, 11]. Our findings indicate that the Norwegian version is both valid and reliable, thus representing an important supplement for measuring subjective outcomes in patients who have sustained PE. Klok et al. [11] established a hypothesized dimensional structure, consisting of six dimensions, which concur with a majority of questions in the questionnaire (except from questions two and three). Basing the construct of a questionnaire merely on a postulated structure may be debateable, since no consideration is made about the underlying structure. Even though it might be assumed that the items are highly correlated, such a structure can only be confirmed and tested by performing a factor analysis. Formal factor analysis consequently plays a major role in construct validation [17]. Deciding which factors to retain is crucial.
123
Table 4 Median values and floor effects of factors extracted in the Norwegian PEmb-QoL PEmb factors
Median
Interquartile range
Floor effects (%)
F1
1.5
1.2–2.2
16
F2
1.4
1.1–2.0
20.2
F3
1.4
1.0–2.2
26.3
F4
1.6
1.0–2.4
33.8
F5
1.7
1.7–2.7
28.6
F6
0.0
0.0–1.0
61.5
PEmb-QoL pulmonary embolism quality of life, F1–F6 extracted factors 1–6 based on principle component analysis
Over- or under-extraction may have deleterious effects on the results [20, 21]. It is debated which method to be used in the factor-retaining process [22], but generally it is
Qual Life Res (2015) 24:417–425
423
Table 5 Test–retest reliability of the Norwegian version of the PEmb-QoL questionnaire (n = 146) Cronbach’s alpha
ICC?
p value
1.4 (0.58)
0.93
0.86
\0.001
1.3 (0.75)
0.88
0.79
\0.001
1.4 (0.49) 1.6 (0.96)
1.4 (0.46) 1.4 (0.90)
0.89 0.90
0.81 0.83
\0.001 \0.001
F5
1.7 (1.07)
1.7 (0.97)
0.86
0.75
\0.001
F6
0.0 (0.60)
0.0 (0.61)
0.88
0.78
\0.001
PEmb N factors
Median (SD) Baseline
Retest
F1
1.5 (0.60)
F2
1.4 (0.79)
F3 F4
PEmb-QoL pulmonary embolism quality of life, ICC intraclass correlation coefficient, ? two-way mixed model—single measures, SD standard deviation
recommended that eigenvalue [1 should be accompanied by another method, e.g., scree plot [20]. However, retaining factors based on only eigenvalue is still an extensively applied approach [17, 23]. In the original article by Klok et al. [11], it is not stated which method was used. The principal component analysis in the current study yielded six factors explaining 71 % of the cumulative variance. The varimax rotation clustered the items fairly consistent with the original dimensional structure, except for factors five and six. Factor five loaded for items 1e, 1g, and 9c, respectively. The first two of these items originate from the frequency of complaints dimension and the last one from the emotional complaints dimension. A logical explanation may be that these questions are interpreted as if the disease has not been fully recovered. Factor six had its highest loading in item 4a, which originate from the ADL limitations dimension, and it could have been expected to load in factor three, clustering items about work-related problems. The reason for this is not clear, but a potential explanation may be that 4a is the only item addressing work per se, which of course not necessarily applies to a large amount of the patients in this population. The latter is underscored by the fact that 61.5 % of patients included in this study reported that they did not work. Moreover, it is well known from studies performed in different patient populations [24] that the number of factor in the same questionnaire may differ across various studies in different countries. Furthermore, discrepancies may be found related to which items that relates to the various factors. The same phenomenon was seen in this study, reproducing the same amount of factors as in the original validation, but with a different item loading. It might be speculated whether observed differences in factorial structure and score may be related to potential differences in clinical and sociodemographic variables between the current study and the study by Klok et al. [11]. Compared to the study by Klok et al. [11], our study had more patients included (n = 213 vs. n = 90), less patients had active
malignancy and less patients were obese. Patients in our cohort were, however, slightly older at the time of inclusion compared to patients in the study by Klok et al. [11] (60 vs. 56 years). Even though not directly comparable, we cannot exclude that these factors may potentially have contributed to the observed differences. There are obvious problems related to such findings. In particular, it may limit the possibility of comparison across studies and boundaries. A possible solution to circumvent this problem may be to calculate and compare a PEmb-QoL sum score. The sum score was, however, not reported in the original validation [11]. It may also be debateable to merely report the sum score on the basis that HRQoL is generally viewed as a multidimensional construct [17]. Sample size is much debated in exploratory factor analysis, and there is no unanimous understanding or presented algorithms for adequate sample size [25, 26]. Strict rules about sample size have mostly disappeared [20, 27]. Traditionally, it has been considered that ‘‘the more the better’’ [25]. Several guidelines mention the subject to item ratio as a common method to be used [28]. Our subject to item ratio was 5.5:1, which is a ratio that is advocated by some and commonly used in several other studies [23, 29]. Internal consistency reliability showed excellent results, way over the recommended limit [24]. However, an excellent Cronbach’s alpha may be a marker for redundancy [21]. In other words, one or more items are basically asking the same question or at least the subject is interpreting the questions in the same way. Very high Cronbach’s alpha can also be found in scales with large number of items, because it is dependent upon the number of items in a scale [27]. In order to investigate whether the PEmbQoL dimensions displayed ambiguous multicollinearity, both variance inflation factor (VIF) and tolerance were calculated. No questionable values were displayed indicating that no problematic multicollinearity was found. There is no general consensus as to how long the between-assessment time gap should be when measuring test–retest reliability. It is generally agreed, however, that a too short test–retest period can result in patients’ recalling their answers from baseline, and a too long period may allow true change in the patients’ clinical condition [17]. In our study, questionnaires were sent to patients in order to calculate test–retest reliability after 2 weeks, meaning that patients completed them at a minimum of 14 days after our initial assessment. In comparison with, e.g., cancer where one would anticipate HRQoL to vary over time, the risk of such variations in the present population was viewed to be low. Furthermore, a retest interval of minimum 14 days was judged as adequate to avoid a recall bias. Findings indicated that the test–retest reliability was good with ICC values consistently over 0.70, which is recommended by some guidelines [27, 30]. Thus, the questionnaire seems to
123
424
have adequate reproducibility and consequently suitable for longitudinal comparisons. Criterion validity was confirmed by rather good correlations between the PEmb-QoL factors and the EQ-5D-3L dimensions. Factors one, three, five, and six (related to ADL and work/regular activity) correlated with EQ-5D3L’s dimension for usual activities. Factor four (pain) correlated significantly with pain in EQ-5D-3L. Finally, factor two (emotional complaints) correlated significantly with anxiety in EQ-5D-3L. The main limitations of the study are the retrospective patients’ identification and the incomplete participation of the original cohort. One might speculate that higher percentage of patients with both reduced health-related quality of life and increased symptom burden agreed to participate. However, given the fact that a marked floor effect was observed in five out of six factors (indicating minor HRQoL problems), this seems unlikely. Problematic floor effects were also reported by Klok et al. [11], and the most plausible explanation seems to be related to the fact that patients in the current study were recruited from a hospital registry, with a median time since diagnosis being 3.6 years. Consequently, a majority of these patients may no longer have troubling symptoms. Furthermore, patients did not rate their symptoms at the two measurement times, which would have been valuable in order to calculate responsiveness (sensitivity to change). Indeed, responsiveness is an indicator of the ability of a questionnaire to detect changes over time and should consequently be evaluated in longitudinal studies [17]. Given the rather long time since the diagnosis of PE, real changes in patients’ health condition may have occurred at an earlier stage, and it is difficult to know whether a symptom score would in fact be related to the PE diagnosis. Moreover, the potential improved level of function over time may have resulted in a response shift among these patients [31]. On the other hand, since this study had a fairly large sample size and inclusion criteria were broad, we judge this cohort to be a representative sample of prior PE patients. We recommend, however, that floor effects and sensitivity to change should be further investigated in future longitudinal follow-up studies of patients with PE. In the current study, EQ-5D-3L was the chosen generic HRQoL instrument. This choice was based on the brevity of the EQ-5D-3L and consequently the potential positive influence this may have on patients’ compliance and completeness of scores. In retrospect, however, the Short-Form 36 (SF-36) might have been more a better choice because of its comprehensiveness for measuring generic HRQoL. There are also some limitations attached to the translation procedure of the PEmb-QoL. Indeed, response choice ratings could have been used in order to test the conformity of the response choices to their hypothesized ordinality. Besides, even though no
123
Qual Life Res (2015) 24:417–425
differences were seen related to response rates when distributing questionnaires either by post or e-mail, questions may be raised related to potential influence on validation. In our view, this seems to be a minor problem, given the fact that questionnaires (regardless of receiving them by post or e-mail) were completed at home. Consequently, having some patients filling them out in the hospital and some at home would have constituted a greater problem. In conclusion, the results from the psychometrical testing of the PEmb-QoL in this Norwegian cohort are promising and indicate overall good reliability, validity, and reproducibility; however, further studies are warranted to evaluate the questionnaire with regard to sensitivity to change (responsiveness) and potential response shifts [32]. Currently, there are no longitudinal follow-up studies from the time of diagnoses of PE using disease-specific measures. Such studies are warranted as well as the need to test the PEmb-QoL in studies aiming to evaluate various treatment effects and interventions.
References 1. Kearon, C. (2003). Natural history of venous thromboembolism. Circulation, 107(23 Suppl 1), I22–I30. 2. Carson, J. L., Kelley, M. A., Duff, A., Weg, J. G., Fulkerson, W. J., Palevsky, H. I., et al. (1992). The clinical course of pulmonary embolism. New England Journal of Medicine, 326(19), 1240–1245. 3. Goldhaber, S. Z., Visani, L., & De Rosa, M. (1999). Acute pulmonary embolism: Clinical outcomes in the International Cooperative Pulmonary Embolism Registry (ICOPER). The Lancet, 353(9162), 1386–1389. 4. Pengo, V., Lensing, A. W., Prins, M. H., Marchiori, A., Davidson, B. L., Tiozzo, F., et al. (2004). Incidence of chronic thromboembolic pulmonary hypertension after pulmonary embolism. New England Journal of Medicine, 350(22), 2257–2264. 5. Klok, F. A., Tijmensen, J. E., Haeck, M. L., van Kralingen, K. W., & Huisman, M. V. (2008). Persistent dyspnea complaints at long-term follow-up after an episode of acute pulmonary embolism: Results of a questionnaire. European Journal of Internal Medicine, 19(8), 625–629. 6. Klok, F. A., van Kralingen, K. W., van Dijk, A. P., Heyning, F. H., Vliegen, H. W., Kaptein, A. A., et al. (2010). Quality of life in long-term survivors of acute pulmonary embolism. Chest, 138(6), 1432–1440. 7. van Es, J., den Exter, P. L., Kaptein, A. A., Andela, C. D., Erkens, P. M., Klok, F. A., et al. (2013). Quality of life after pulmonary embolism as assessed with SF-36 and PEmb-QoL. Thrombosis Research, 132(5), 500–505. 8. Hawker, G., Melfi, C., Paul, J., Green, R., & Bombardier, C. (1995). Comparison of a generic (SF-36) and a disease specific (WOMAC) (Western Ontario and McMaster Universities Osteoarthritis Index) instrument in the measurement of outcomes after knee replacement surgery. Journal of Rheumatology, 22(6), 1193–1196. 9. Salaffi, F., Carotti, M., & Grassi, W. (2005). Health-related quality of life in patients with hip or knee osteoarthritis: Comparison of generic and disease-specific instruments. Clinical Rheumatology, 24(1), 29–37.
Qual Life Res (2015) 24:417–425 10. Cohn, D. M., Nelis, E. A., Busweiler, L. A., Kaptein, A. A., & Middeldorp, S. (2009). Quality of life after pulmonary embolism: the development of the PEmb-QoL questionnaire. Journal of Thrombosis and Haemostasis, 7(6), 1044–1046. 11. Klok, F. A., Cohn, D. M., Middeldorp, S., Scharloo, M., Buller, H. R., van Kralingen, K. W., et al. (2010). Quality of life after pulmonary embolism: Validation of the PEmb-QoL questionnaire. Journal of Thrombosis and Haemostasis, 8(3), 523–532. 12. Beaton, D. E., Bombardier, C., Guillemin, F., & Ferraz, M. B. (2000). Guidelines for the process of cross-cultural adaptation of self-report measures. Spine (Phila Pa 1976), 25(24), 3186–3191. 13. EuroQol, G. (1990). EuroQol—A new facility for the measurement of health-related quality of life. Health Policy, 16(3), 199–208. 14. Brooks, R. (1996). EuroQol: the current state of play. Health Policy, 37(1), 53–72. 15. Cronbach, L. J. (1951). Coefficient alpha and the internal structure of tests. Psychometrika, 16(3), 297–334. 16. DeVellis, R. F. (1991). Scale development: Theory and applications (1st ed.). Newbury Park, CA: Sage. 17. Fayers, P. M., & Machin, D. (2007). Quality of life : The assessment, analysis and interpretation of patient-reported outcomes (2nd ed.). Chichester: Wiley. 18. Hocking, R. R. (2003). Methods and applications of linear models: Regression and the analysis of variance. Hoboken, New Jersey: Wiley. 19. McHorney, C. A., Ware, J. E, Jr, Lu, J. F., & Sherbourne, C. D. (1994). The MOS 36-item Short-Form Health Survey (SF-36): III. Tests of data quality, scaling assumptions, and reliability across diverse patient groups. Medical Care, 32(1), 40–66. 20. Costello, A. B., & Osborne, J. W. (2005). Best practices in exploratory factor analysis: Four recommendations for getting the most from your analysis. Practical Assessment, Research & Evaluation, 10, 173–178. 21. Pesudovs, K., Burr, J. M., Harley, C., & Elliott, D. B. (2007). The development, assessment, and selection of questionnaires. Optometry and Vision Science, 84(8), 663–674.
425 22. Zwick, W. R., & Velicer, W. F. (1986). Comparison of five rules for determining the number of components to retain. Psychological Bulletin, 99(3), 432. 23. De Vet, H. C. W., Ade`r, H. J., Terwee, C. B., & Pouwer, F. (2005). Are factor analytical techniques used appropriately in the validation of health status questionnaires? A systematic review on the quality of factor analysis of the SF-36. Quality of Life Research, 14(5), 1203–1218. 24. Aaronson, N., Alonso, J., Burnam, A., Lohr, K. N., Patrick, D. L., Perrin, E., et al. (2002). Assessing health status and quality-of-life instruments: Attributes and review criteria. Quality of Life Research, 11(3), 193–205. 25. Floyd, F. J., & Widaman, K. F. (1995). Factor analysis in the development and refinement of clinical assessment instruments. Psychological Assessment, 7(3), 286–299. 26. Guadagnoli, E., & Velicer, W. F. (1988). Relation of sample size to the stability of component patterns. Psychological Bulletin, 103(2), 265–275. 27. Terwee, C. B., Bot, S. D., de Boer, M. R., van der Windt, D. A., Knol, D. L., Dekker, J., et al. (2007). Quality criteria were proposed for measurement properties of health status questionnaires. Journal of Clinical Epidemiology, 60(1), 34–42. 28. Streiner, D. L. (1994). Figuring out factors: The use and misuse of factor analysis. Canadian Journal of Psychiatry, 39(3), 135–140. 29. Kline, P. (1993). The handbook of psychological testing (2nd ed.). London: Routledge. 30. Nunnally, J. C., & Bernstein, I. H. (1994). Psychometric theory (3rd ed.). New York: McGraw-Hill. 31. Calvert, M., Blazeby, J., Altman, D. G., Revicki, D. A., Moher, D., Brundage, M. D., et al. (2013). Reporting of patient-reported outcomes in randomized trials: The CONSORT PRO extension. JAMA, 309(8), 814–822. 32. Schwartz, C. E., Andresen, E. M., Nosek, M. A., Krahn, G. L., & Measurement, R. E. P. o. H. S. (2007). Response shift theory: Important implications for measuring quality of life in people with disability. Archives of Physical Medicine and Rehabilitation, 88(4), 529–536.
123
