Journal of Psychosomatic Research 79 (2015) 43–48
Contents lists available at ScienceDirect
Journal of Psychosomatic Research
The dimensional structure of the MacNew Health Related Quality of Life questionnaire: A Mokken Scale Analysis O. Friedrich a,⁎, J. Sipötz a,b, W. Benzer c, E. Kunschitz a,b, S. Höfer d a
Karl Landsteiner Institute for Scientific Research in Clinical Cardiology, Vienna, Austria Department of Cardiology, Hanusch Krankenhaus, Vienna, Austria Department of Interventional Cardiology, Academic Hospital, Feldkirch, Austria d Department of Medical Psychology, Innsbruck Medical University, Austria b c
a r t i c l e
i n f o
Article history: Received 9 February 2015 Received in revised form 16 April 2015 Accepted 17 April 2015 Keywords: MacNew Mokken Scale Analysis Health related quality of Life Coronary Artery Disease
a b s t r a c t Objective: The MacNew Health related Quality of Life Questionnaire is a widely used instrument for the assessment of health related quality of life in cardiac patients. The study addresses for the first time the dimensional structure of the MacNew with Mokken Scale Analysis (MSA). Methods: Separate exploratory MSA of the MacNew was conducted in a large Spanish (n = 1012) and a medium sized Austrian sample (n = 262) of patients with Coronary Artery Disease (CAD) after Percutaneous Coronary Intervention (PCI). The results of both samples were summarized in a synthesis model. Confirmatory MSA and Confirmatory Factor Analysis (CFA) were used to evaluate the model. Results: The synthesis model comprises 21 items forming a unidimensional sum scale of moderate strength. On the level of subdomains we define two strong unidimensional subscales (restriction: 6 items, and emotional: 10 items) and two smaller item sets (symptoms: 2 items and social: 3 items). 5 items were excluded due to low scalability in both samples. Conclusion: Our results generally support the use of the MacNew Global score, with the limitation, that five items may be questionable with regard to scalability. On the level of unidimensional subscales MSA suggests to differentiate between a six-item restriction scale and a ten-item emotional scale. The study demonstrates that Mokken Scale Analysis complements the results of factor analysis and can contribute to a more comprehensive understanding of the dimensional structure of Health-related Quality of Life questionnaires. © 2015 Elsevier Inc. All rights reserved.
Introduction The MacNew questionnaire is a widely used instrument for disease specific measurement of Health Related Quality of Life (HRQL) in cardiac patients. The questionnaire is originated in the Quality of Life after Myocardial Infarction (QLMI) interview, introduced by Oldridge et al. [1,2] in order to investigate the effects of cardiac rehabilitation after myocardial infarction in anxious and/or depressed patients. The QLMI instrument was generated by a clinimetric approach through interviews with physicians, nurses, health care professionals and patients with myocardial infarction leading to a set of 26 items covering five domains (symptoms, restrictions, confidence, self-esteem and emotional functioning). Based on the QLMI Lim et al. [3] and Valenti et al. [4] developed the patient self-administered MacNew questionnaire. The MacNew contains 27 questions, 24 items of the QLMI and three newly added items, ⁎ Corresponding author at: Karl Landsteiner Institute for Scientific Research in Clinical Cardiology, Hanusch Krankenhaus, Heinrich-Collin-Str. 30, A-1120 Vienna, Austria. Tel.: +43 1 680 212 3203; fax: +43 1 910 21 85219. E-mail address: [email protected] (O. Friedrich).
http://dx.doi.org/10.1016/j.jpsychores.2015.04.007 0022-3999/© 2015 Elsevier Inc. All rights reserved.
rated by a seven point Likert scale. The MacNew measures HRQL in three domains (emotional, physical, and social). A global scale can be calculated by summing up all items [4]. The MacNew has been made available in over 80 countries and validated in over 16 languages [5] in patients with a variety of cardiac conditions ranging from myocardial infarction to angina, acute coronary syndrome, heart failure and in patients with a pacemaker and is recommended as a core heart disease quality of life instrument [6]. The three-dimensional factor structure of the MacNew, originally proposed by Valenti et al. [4] for the Australian English version on the basis of Principal Component Analysis (PCA) is characterized by some ambiguity. About half of the items (12/26 — one item was excluded from analysis) showed substantial cross-loadings and were therefore allocated to more than one subscale. The three-factor solution was generally accepted for different language versions of the MacNew even if the results of factor analysis differed sometimes partly, especially in smaller samples and in particular with regard to items of the physical and social subscales [6]. In order to analyse HRQL on the level of the subdomains a unidimensional factor structure would be highly preferable. However, given the nature of HRQL (being multidimensional and
44
O. Friedrich et al. / Journal of Psychosomatic Research 79 (2015) 43–48
substantially interrelated) and the published results, the attempt to define such scales by allocating items according item loadings in factor analysis seems problematic. Our analysis aimed to readdress the factor structure of the German and Spanish MacNew questionnaire versions with a different approach by using Mokken Scale Analysis (MSA). MSA is based on the principles of Item Response Theory (IRT). It assumes that the probability of a given response is influenced by a latent trait (e.g. an individual's ability and item difficulty), and hence enables the evaluation of the scalability of single items with respect to this latent trait. Unlike parametric IRT techniques MSA does not require the assumption of a parametric form of item response functions [7,8]. MSA is particularly useful in order to investigate the dimensionality of scales. By employing a “bottom up” clustering search procedure using preselected cutoff points for item scalability, MSA allows, unlike other techniques, analysis of the dimensional structure of a scale on different hierarchical levels [9]. Our work comprises two parts. First, we conducted separate exploratory MSA of the MacNew in a large Spanish and a medium sized Austrian sample of patients with Coronary Artery Disease (CAD) undergoing Percutaneous Coronary Intervention (PCI). Second, we combined the results of both samples to a synthesis model, evaluated this model by confirmatory MSA and Confirmatory Factor Analysis (CFA) and compared it to existing models. Methods Participants
the dimensional structure on different hierarchical levels we employed a step by step approach starting with the established cutoff value for Hj (c = 0.3) and subsequently increasing by steps of 0.05 up to c = 0.6 [8]. Confirmatory MSA was conducted by calculating homogeneity coefficients of preselected item sets. Violations of monotonicity were assessed via CRIT statistic with values b 40 considered to be acceptable [11]. MSA was performed with R 3.0.3 and the R package Mokken [12,13]. The lower bound of scale reliability was estimated with Cronbach's α. Following recommendations with regard to Likert-type data we additionally calculated ordinal alpha, which is based on polychoric correlations instead of Pearson correlations and provides a more accurate estimate of reliability for our data [14]. Confirmatory Factor Analysis (CFA) CFA was executed in R 3.0.3 with the R package lavaan [15] and the graphical Structural Equation Modelling software Ωnyx [16]. Model fit was estimated by Comparative Fit Index (CFI), Tucker–Lewis Index (TLI) and root mean square error of approximation (RMSEA). CFI and TLI values N 0.90 indicate good fit, RMSEA b 0.08 suggest moderate, RMSEA b 0.06 good fit [17]. Because Mardia testing indicated nonnormal multivariate distribution we calculated robust indices with Satorra–Bentler correction [18]. Measurement errors of items within subscales were allowed to intercorrelate by referring to modification indices when appropriate. Results
The study sample comprised 1082 Spanish and 310 Austrian patients from the PRODES Xience Stent registry treated with PCI in 40 Spanish and 7 Austrian centres. The PRODES registry was designed to investigate changes in Health related Quality of Life and Mental Distress after PCI by a prospective, multi-centre approach. Patients with acute ST-elevation myocardial infarction (STEMI) or stent deployment during the last 6 months were excluded from the registry. The data were collected between January 1st, 2008 and December 31st, 2011. All Patients answered the MacNew questionnaire during hospital stay after PCI and agreed to participate by signing an informed consent. The registry was conducted in accordance to the Helsinki Declaration and the guidelines of Good Clinical Practice after approval by the ethical committee of the City of Vienna (EK-07-202-VK). Part of the Austrian data was previously published in another context elsewhere [10]. Mokken Scale Analysis (MSA) MSA assumes that the data fit three basic requirements constituting the Monotone Homogeneity Model (MHM): First, there are unidimensional latent traits capturing the association of item scores. Second, Item Response Curves are monotonically nondecreasing. Third, responses of one subject to the different items are stochastically independent [7]. If these assumptions are met item sum scores can be used to characterize and compare subjects with regard to a latent trait. MSA uses the Loevinger homogeneity coefficient Hj to describe the strength of the relationship between an item and the latent trait with high values suggesting a good ability to discriminate between low and high scores of the trait. Hj values ≥ 0.3 indicate scalability of an item. The total scale coefficient H reflects the discrimination power of sets of items constituting a scale. As a rule of thumb 0.3 ≤ H b 0.4 defines a weak scale, 0.4 ≤ H b 0.5 a moderate scale and H ≥ 0.5 a strong scale [7]. The scale search procedure in explanatory MSA follows a bottom-up approach. It starts off with the pair of items with the highest homogeneity coefficient Hj, and then stepwise adding items with maximizing the scale homogeneity coefficient H. The algorithm proceeds until no item exceeding a predefined cutoff value for Hj is left. Subsequently a new scale is formed following the same procedure. The process is finished when no items fulfilling the criterion of inclusion are left. To describe
Study sample The initial study samples comprised 1082 Spanish and 310 Austrian patients answering the MacNew questionnaire during hospital stay after PCI. The percentage of missing values of items 1 to 26 was between 0.9% and 2.6%. Item 27 (sexual intercourse) was skipped or answered with the “not applicable” response by 36% of participants. Because the MSA procedure requires complete datasets, we excluded all questionnaires with missing values for items 1–26. Item 27 was omitted from analysis. The final samples consisted of 1012 Spanish and 262 Austrian patients. Patient characteristics are summarized in Table 1. Exploratory Mokken Scale Analysis The results of the separate MSA for the Spanish and the Austrian sample are shown in Table 2. In general homogeneity coefficients were higher in the Austrian sample. With the cutoff value for Hj c = 0.3 in both samples a sumscale of moderate strength (Spain: H = 0.41, Austria: H = 0.44) was found. Item 3 of the Spanish sample and item 22 of the Austrian sample were unscalable (Hj b 0.3).
Table 1 Patient characteristics.
Age ± SD Male [%] Smoking [%] Hypercholesterolemia [%]a Hypertonia [%]a IDDM [%]a Obesity (BMI ≥ 30) [%] CCS classification [%]
Single vessel intervention [%] AHA classification [%]
CCS 0/I CCS II CCS III CCS IV AHA A AHA B AHA C
Spain (n = 1012)
Austria (n = 262)
62.7 ± 10.4 68.1 40.5 71.9 52.1 9.1 31.6 20.8 33.4 32.2 13.6 84.0 7.1 70.8 22.1
62.9 ± 9.4 71.4 33.9 87.5 70.2 7.9 26.7 15.1 52.3 23.4 9.2 81.0 9.7 62.6 27.7
IDDM: Insulin Dependent Diabetes Mellitus; CCS: Canadian Cardiovascular Society classification of angina pectoris; AHA American Heart Association classification of cardiac stenosis. a Treated.
O. Friedrich et al. / Journal of Psychosomatic Research 79 (2015) 43–48
45
Table 2 Separate explanatory MSA of the Spanish and the Austrian sample. Figures represent Loevinger item homogeneity coefficients (Hj) and scale homogeneity coefficients (H). Matching results highlighted in bold. S1: scale 1, S2: scale 2, S3: scale 3, S4: scale 4. Spain: N = 1012; Austria: N = 262. Item
Spain c = 0.3
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26
Frustrated Worthless Confident Down in the dumps Relaxed Worn out Happy Restless Short of breath Tearful More dependent Social activities Others/less confidence Chest pain Lack self-confidence Aching legs Sports/exercise limited Frightened Dizzy/lightheaded Restricted or limited Unsure about exercise Overprotective family Burden on others Excluded Unable to socialize Physically restricted H
Austria c = 0.50
c = 0.3
c = 0.55 S1
Sumscale
S1
S2
S3
S4
Sumscale
0.35 0.45 – 0.47 0.40 0.46 0.40 0.43 0.40 0.40 0.43 0.44 0.32 0.36 0.49 0.36 0.44 0.46 0.38 0.47 0.40 0.30 0.41 0.44 0.44 0.47 0.41
–
– 0.53 – 0.59 0.52 0.50 0.50 0.54
–
–
–
–
0.47 0.46 0.36 0.48 0.42 0.48 0.49 0.44 0.35 0.42 0.49 0.49 0.37 0.44 0.51 0.34 0.45 0.49 0.34 0.50 0.40 – 0.45 0.45 0.48 0.48 0.44
–
0.51 0.50 0.51 –
– 0.63 – 0.66 0.53 – 0.57 0.53 0.66 0.60
–
– 0.51
–
0.55 –
–
–
0.51 –
–
–
–
–
– 0.51
0.53
The scale search procedure revealed two strong subscales (H N 0.5) and two smaller item sets with the cutoff values for Hj c = 0.50 (Spain) and c = 0.55 (Austria): • In both samples scale 1 included six items of the physical/social domain (17, 20, 21 and 24–26). • Scale 2 consisted of items of the emotional domain: 9 items in the Spanish sample (2, 4–8, 10, 15, 18) and 8 items in the Austrian sample (1, 2, 4–8, 15). • Scale 3 in the Spanish sample comprised two items describing physical symptoms (9 and 14), which were excluded in the Austrian sample because of Hj b 0.55. With exception of the excluded item 23 (Hj b0.50), scale 4 of the Spanish sample corresponded to scale 3 of the Austrian sample, which combined items of the social domain (11, 12, 23). Scale 4 of the Austrian sample included two items of the emotional domain (10 and 18), which were part of scale 1 in the Spanish sample. • Five items (3, 13, 16, 19, 22) were excluded in both samples because of Hj b c at the respective levels.
Internal consistency was adequate for the sumscales and the subscales S1 and S2 in both samples (Cronbach's α ≥ 0.8, ordinal α ≥ 0.9). Lower reliability was found in the two smaller item sets S3 and S4 (Table 3). Synthesis model Combining the results of separate explanatory MSA at the cutoff values for Hj c = 0.50 (Spain) and c = 0.55 (Austria) we created a model proposal consisting of a restriction subscale with items covering restriction to physical or social activity connected to the heart problem, and an emotional subscale. In addition we defined two smaller sets (symptoms and social) including the items with higher discrimination power not assigned to either of the two subscales: • The restriction subscale includes the six physical/social items (17, 20, 21 and 24–26), which constituted scale 1 in both samples.
0.51
0.51
S2 0.64 0.61 – 0.64 0.55 0.55 0.60 0.58 –
–
–
– – – 0.64 – 0.67 0.59 – 0.57 0.59 0.66 0.62
S3
S4
–
–
–
– 0.57
– – 0.58 –
0.58 0.57 – –
– –
–
–
–
–
0.57 –
–
– 0.56
–
0.59
0.57
0.57
• The emotional subscale comprises ten items. Beside emotional items 2, 4–8, 15, which are part of scale 2 in both samples, we included item 1, which was missing in the Spanish sample because of Hj b c, and items 10 and 18, which constituted a own scale fragment in the Austrian sample, closely related to scale 2. • The symptom item set includes two items and corresponds to scale 3 in the Spanish sample with items 9 and 14 addressing symptoms of CAD (angina pectoris and dyspnoea). In the Austrian sample these items were not part of an item set at c = 0.55. • The social item set comprises items 11, 12, and 23 and corresponds to scale 3 of the Austrian sample and scale 4 of the Spanish sample with the single exception of item 12, which was missing in the Spanish sample because of Hj b 0.50. • The five items 3, 13, 16, 19, and 22 with lower discrimination power, not designated to item aggregates in either sample at c = 0.50 (Spain) and c = 0.55 (Austria), were excluded.
To ensure the created model fits the data of both samples we conducted confirmatory MSA for the Spanish and the Austrian sample separately. The results for the 4 subscales and the sumscale are summarized in Table 4. Homogeneity coefficients indicated high (H ≥ 0.5) discrimination power for the restriction and the emotional subscales and at least moderate discrimination power (H ≥ 0.4) for the two smaller sample sets in both samples. Because of excluding weaker items the moderate sumscale has gained strength (Spain: from 0.41 to 0.45; Austria: from 0.44 to 0.48). Reliability was high for the sumscale and both subscales (restriction and emotional), but not for the two smaller scale fragments (symptoms and social) (Table 5). Confirmatory Factor Analysis confirmed adequate fit of the sumscale and the 4-factor structure of synthesis model for the two samples separately and for both combined (Table 6). The total variance explained was 50.5% for the Spanish sample (restriction: 16.2%, emotional: 23.1%, symptoms: 4.7%, social: 6.6%), 53.3% for the Austrian sample (restriction: 16.6%, emotional: 25.0%, symptoms: 4.3%, social: 7.4%) and 50.8% for both samples combined (restriction: 16.2%, emotional: 23.3%, symptoms: 4.6%, social: 6.7%). Factor
Table 3 Reliability of scales resulting from explanatory MSA of the Spanish and the Austrian sample. Spain
Cronbach's α Ordinal α # items
Austria
Sumscale
S1
S2
S3
S4
Sumscale
S1
S2
S3
S4
0.94 0.95 25
0.88 0.91 6
0.90 0.91 9
0.66 0.68 2
0.64 0.72 2
0.94 0.96 25
0.89 0.92 6
0.91 0.92 8
0.76 0.82 3
0.66 0.72 2
46
O. Friedrich et al. / Journal of Psychosomatic Research 79 (2015) 43–48
Table 4 Separate confirmatory MSA of the synthesis model for the Austrian and the Spanish sample. Figures represent Loevinger item homogeneity coefficients (Hj) and scale homogeneity coefficients (H). Spain: N = 1012; Austria: N = 262. Item
Restriction Spain
1 2 4 5 6 7 8 9 10 11 12 14 15 17 18 20 21 23 24 25 26
Frustrated Worthless Down in the dumps Relaxed Worn out Happy Restless Short of breath Tearful More dependent Social activities Chest pain Lack self-confidence Sports/exercise limited Frightened Restricted or limited Unsure about exercise Burden on others Excluded Unable to socialize Physically restricted H
Emotional Austria
Symptoms
Spain
Austria
0.48 0.53 0.59 0.52 0.49 0.49 0.54
0.59 0.61 0.53 0.53 0.58 0.57 0.52
0.50
0.58
Spain
0.51
0.63
0.64
0.63 0.53
0.67 0.59
0.57 0.53 0.66 0.60
0.57 0.59 0.66 0.62
0.52
0.50
0.59
0.52
0.57
loadings and correlations between latent variables of the synthesis model are summarized in Fig. 1.
Discusssion The separate Mokken Scale Analysis of the Spanish and the Austrian sample led to very similar results and allowed for proposing a combined model of the unidimensional structure of the MacNew Health related Quality of Life Questionnaire. The model suggests two strong unidimensional subscales: The restriction subscale comprises all six items, which were designated to both, the physical and the social scale, in the original factor structure proposal by Valenti et al. [4]. These items are characterized by drawing a causal relationship between the term “heart problem” (“Herzproblem”, “problema de corazón”) and restrictions in physical or social activities (Items 17, 20, 24–26) or in case of item 21 addressing restriction in sports activity directly. The emotional subscale corresponds largely to the emotional subscale of the initially proposed three-factor model. It includes all eight items assigned exclusively to the emotional domain (1, 4–8, 10, 18) plus two items assigned to both the emotional and the social domain (2 and 15). In addition we defined two item sets comprising mainly items of the physical and the social subscale of the original factor structure proposal. Because of the small number of items (two and three, respectively), these aggregates showed lower levels of reliability and cannot legitimately be considered as fullfledged subscales. Nevertheless they may be of interest, since they contribute to a better understanding of the heterogeneity in the social and physical subscale. Two items exclusively designated to the physical subscale by Valenti et al. form an aggregate addressing the symptom status of CAD (angina pectoris and dyspnoea). Unlike the items of the restriction subscale these two questions are not specifically related to restriction of social or physical activities. Interestingly the two other
Spain
Sumscale Austria
0.48
0.51 0.54
Social Austria
0.51 0.47
0.58 0.57
0.47
0.56
0.48
0.57
0.48
0.51
0.48
Spain
Austria
0.37 0.47 0.49 0.42 0.47 0.42 0.45 0.41 0.41 0.44 0.46 0.37 0.51 0.45 0.47 0.50 0.41 0.42 0.46 0.46 0.49 0.45
0.48 0.48 0.49 0.43 0.49 0.51 0.46 0.35 0.44 0.51 0.51 0.45 0.53 0.48 0.51 0.53 0.43 0.46 0.48 0.50 0.52 0.48
exclusively physical items in the Valenti analysis (item 16: aching legs and item 19: feeling dizzy/lightheaded) were not part of this aggregate. This could possible be due to the fact, that both describe somatic symptoms, which are far less specific — at least with regard to CAD patients. The second item set includes three items covering dependence on others (items 11 and 23) and inability to meet social obligations in the usual way (item 12). In the original three-factor proposal these items were assigned to the social subscale exclusively (item 11), to all three subscales (item 12) and to the social and emotional subscale (item 23). They can be considered as a remainder of the social scale with items fitting neither the emotional nor the restriction subscale of our model. Five items (3, 13, 16, 19 and 22) were excluded because they showed low discrimination power and failed to build item sets in both samples. Overall our MSA is largely in line with the results of PCA reported by Valenti et al. [4] in essential respects. The two large blocks of items with similar loading patterns of this analysis – physical plus social restrictions and exclusively emotional – were assigned to two separate subscales. The differences between our results and the Valenti proposal can be largely explained by the different methodological approach. The MSA scale search procedure is specifically designed to define unidimensional scales in a hierarchical clustering process with evaluating items by their ability to discriminate with regard to a latent trait. Factor analysis methods in general and PCA in particular are, by contrast, primarily instruments for dimensionality reduction. They make no a priori assumptions about the dimensional structure of an item set and are therefore not particularly well suited to investigate the unidimensionality of item sets [9]. In neither of our both samples separate social and physical subscales were defined, because the scale search procedure first formed the restriction subscale, comprising items with loadings on the physical scale and the social scale, and thereafter the emotional
Table 5 Reliability of sumscale, subscales (restriction, emotional) and scale fragments (symptoms, social) of the synthesis model. Restriction
Cronbach's α Ordinal α # items
Emotional
Symptoms
Social
Sumscale
Spain
Austria
Spain
Austria
Spain
Austria
Spain
Austria
Spain
Austria
0.88 0.91 6
0.89 0.92 6
0.90 0.92 10
0.91 0.93 10
0.66 0.68 2
0.61 0.66 2
0.72 0.78 3
0.76 0.82 3
0.94 0.95 21
0.94 0.96 21
O. Friedrich et al. / Journal of Psychosomatic Research 79 (2015) 43–48
47
Table 6 Fit of synthesis model for the Spanish, the Austrian sample and both samples combined. Sumscalea
4 factors
CFI TLI RMSEA (CI) a
Spain
Austria
Combined
Spain
Austria
Combined
0.93 0.92 0.059 (0.056–0.063)
0.91 0.90 0.072 (0.064–0.079)
0.93 0.92 0.060 (0.057–0.063)
0.97 0.96 0.045 (0.040–0.049)
0.94 0.92 0.062 (0.054–0.071)
0.97 0.95 0.046 (0.043–0.050)
After allowing measurement errors to intercorrelate within subdomains when appropriate.
scale. Besides the small scale fragments, the then remaining items were too heterogeneous to form own domains, which is reflected by the variety of loading patterns of these items in the PCA of Valenti et al. The results of factor analysis and Mokken scaling are not contradictory but rather complement each other: While the loading patterns of factor analysis show how single items cover the three basic dimensions of Health-related Quality of Life (physical, emotional, social) [19], Mokken analysis allows defining separate subscales. The ambiguity of loading patterns in factor analysis of the MacNew reflects the fact that items covering Health-related Quality of Life are by nature not always exclusively assignable to one of the three basic domains of Quality of Life. Mokken analysis enables extracting unidimensional subscales, which can be used to compare distinct aspects of Health-related Quality of Life besides the level of the global score. In the literature on the dimensional structure of the MacNew questionnaire it is a common theme to point out that the model proposed by Valenti et al. is in need for revision, especially with regard to the physical and the social subscale. Critiques partly refer to divergent loading patterns found in validation studies in different language versions of the MacNew, which were also reported for the German [20–23] and the Spanish version [24]. However, such differences have to be evaluated with some caution because they may well be due, at least in part, to sample size issues and/or potential translation biases. The largest
study providing detailed results on a three dimensional structure of the MacNew confirmed the findings of Valenti et al. convincingly. Validating the Hebrew version in 773 patients undergoing CABG surgery, Geulayov et al. [25] recently reported an almost identical loading pattern in PCA with just one noteworthy variation — item 24 (excluded) without substantial cross-loading on the physical domain. We think the main concern with regard to results of factor analysis of the MacNew is not divergence in loading patterns but how to interpret these patterns given their inherent ambiguity. Two authors have proposed alternative models for the factor structure of the MacNew based on factor analysis. Gramm et al. [26] put forward a proposal with four unidimensional factors based on the results of a large German sample (n = 5692) of cardiac rehabilitation patients. He resolved the issue of ambiguous loading patterns in PCA by removing items with substantial cross-loadings in the three-factor structure and included only items with loadings of N 0.5 for one factor and b0.4 for all other factors in his analysis. Dempster et al. [27] proposed a fivefactor solution based on the results of a relatively small sample (n = 117) of patients with ischaemic heart disease from the UK. He achieved his unidimensional – with the exception of item 23, which was assigned to two factors – solution by grouping items with cross-loadings under the factor they loaded heaviest and/or designating items by reasons of content. Both proposals include an emotional subscale corresponding
Fig. 1. Summary of the synthesis model based on the Spanish and the Austrian data combined.
48
O. Friedrich et al. / Journal of Psychosomatic Research 79 (2015) 43–48
largely to the emotional scale of the original three-factor solution and our model. The remaining items were designated to three and four small factors, respectively, comprising between two and five items. We think that both of these proposals are problematic for methodological reasons. Gramm's strategy to omit items with cross-loadings is not based on item scalability and resulted in the exclusion of four of the six items constituting the restriction subscale of our model. Dempster's approach of assigning items according their strongest loading is also not convincing from a methodological point of view, especially given the relatively small sample size. Gramm, who tested this model on a large German sample, found, that it performed even worse than the threefactor model that he rejected in part in his paper. Strengths and limitations The strength of our study is that we for the first time applied an approach of probabilistic test theory to the MacNew. MSA not only allows evaluating the scalability of the items, but also is particularly suited to investigate the unidimensionality of the questionnaire. The study has also some limitations. First and foremost our analysis is confined to the psychometric evaluation of the MacNew exclusively. Further research is needed to prove the practical relevance of our results for clinical purposes, especially of the newly proposed restriction subscale. Second, our two samples comprised CAD patients after PCI only. Since the MacNew is recommended as a core heart disease quality of life instrument and used today in patients with a broad variety of cardiac conditions further research is needed to prove the generality of our results. Finally, a possible objection to our results may be, that our findings are biassed by the fact that participants answered the questionnaire during their hospital stay after PCI, especially with regard to items covering social interaction. To dispel that concern we have applied our model on six months follow up data of the Austrian sample and were able to confirm the findings of the baseline data with confirmatory MSA and CFA. Conclusion Summing up our results generally support the use of the MacNew Global score, with the limitation, that five of the 26 items (3, 13, 16, 19 and 22) may be questionable with regard to scalability. Further analysis including patients with various cardiac conditions has to prove if the latter is a general concern. This may easily be achieved by utilizing existing MacNew datasets in various languages. On the level of unidimensional subscales MSA suggests differentiating between a six-item restriction scale and a ten-item emotional scale. Overall the study demonstrates that in order to identify distinct unidimensional scales it is useful to combine the results of factor analysis with IRT-based methods like MSA. Conflict of interest declaration The authors declare no conflict of interest. Acknowledgements The work was supported by an unrestricted grant from Abbott Vascular (6095).
References [1] Oldridge N, Guyatt G, Jones N, et al. Effects on quality of life with comprehensive rehabilitation after acute myocardial infarction. Am J Cardiol 1991;67:1084–9. [2] Hillers TK, Guyatt GH, Oldridge N, et al. Quality of life after myocardial infarction. J Clin Epidemiol 1994;47:1287–96. [3] Lim LLY, Valenti LA, Knapp JC, et al. A self-administered quality-of-life questionnaire after acute myocardial infarction. J Clin Epidemiol 1993;46:1249–56. [4] Valenti L, Lim L, Heller RF, Knapp J. An improved questionnaire for assessing quality of life after acute myocardial infarction. Qual Life Res 1996;5:151–61. [5] macnew.org. MacNew health-related quality of life instrument. [Internet]. [cited 2015 Jan 1]; available from: http://www.macnew.org. [6] Höfer S, Lim L. The MacNew heart disease health-related quality of life instrument: a summary. Health Qual Life Outcomes 2004;2:10–1186. [7] Van Schuur WH. Mokken scale analysis: between the Guttman scale and parametric item response theory. Polit Anal 2003;11:139–63. [8] Stochl J, Jones PB, Croudace TJ. Mokken scale analysis of mental health and wellbeing questionnaire item responses: a non-parametric IRT method in empirical research for applied health researchers. BMC Med Res Methodol 2012;12:74. [9] Straat JH, van der Ark L, Sijtsma K. Methodological artifacts in dimensionality assessment of the hospital anxiety and depression scale (HADS). J Psychosom Res 2013; 74:116–21. [10] Sipötz J, Friedrich O, Höfer S, Benzer W, Chatsakos T, Gaul G. Health related quality of life and mental distress after PCI: restoring a state of equilibrium. Health Qual Life Outcomes 2013;11:144. [11] Molenaar IW, Sijtsma K. User's manual MSP5 for Windows. Groningen: IEC ProGAMMA; 2000. [12] van der Ark L. Mokken scale analysis in R. J Stat Softw 2007;20. [13] van der Ark L. New developments in Mokken scale analysis in R. J Stat Softw 2012;48. [14] Gadermann AM, Guhn M, Zumbo BD, Columbia B. Estimating ordinal reliability for Likert-type and ordinal item response data: a conceptual, empirical, and practical guide. Pract Assessment Res Eval 2012;17:1–12. [15] Rosseel Y. lavaan: an R package for structural equation modeling. J Stat Softw 2012;48. [16] Von Oertzen T, Brandmaier AM, Tsang S. Structural equation modeling with Ωnyx. Struct Equ Model A Multidiscip J 2014:1–14. [17] Hu L, Bentler PM. Cutoff criteria for fit indexes in covariance structure analysis: conventional criteria versus new alternatives. Struct Equ Model A Multidiscip J 1999;6: 1–55. [18] Curran P, West S, Finch J. The robustness of test statistics to nonnormality and specification error in confirmatory factor analysis. Psychol Methods 1996;1:16–29. [19] The WHOQOL Group. The World Health Organization quality of life assessment (WHOQOL): position paper from the World Health Organization. Soc Sci Med 1995;41:1403–9. [20] Höfer S, Benzer W, Brandt D, Laimer H, Schmid P, Bernardo A. MacNew Heart Disease Lebensqualitätsfragebogen nach Herzinfarkt: Die deutsche version. Z Klin Psychol Psychother 2004;33:270–80. [21] Höfer S, Anelli-Monti M, Berger T, Hintringer F, Oldridge N, Benzer W. Psychometric properties of an established heart disease specific health-related quality of life questionnaire for pacemaker patients. Qual Life Res 2005;14:1937–42. [22] Höfer S, Schmid J, Frick M. Psychometric properties of the MacNew heart disease health‐related quality of life instrument in patients with heart failure. J Eval Clin Pract 2008;14:500–6. [23] Höfer S, Saleem A, Stone J, Thomas R, Tulloch H, Oldridge N. The MacNew heart disease health-related quality of life questionnaire in patients with angina and patients with ischemic heart failure. Value Health 2012;15:143–50. [24] Cuixart Brotons C, Ribera Solé A, Permanyer Miralda G, et al. Adaptacion del cuestionario de calidad de vida postinfarto MacNew QLMI para su uso en la poblacion espanola. Med Clin 2000;115:768–71. [25] Geulayov G, Oldridge NB, Ziv A, Novokov I, Drory Y, Dankner R. Validation of the Hebrew version of the MacNew heart disease healthrelated quality of life questionnaire in patients undergoing coronary artery bypass surgery. J Clin Exp Cardiolog 2013;04. [26] Gramm L, Farin E, Jaeckel WH. Psychometric properties of the German version of the MacNew heart disease health-related quality of life questionnaire. Health Qual Life Outcomes 2012;10:83. [27] Dempster M, Donnelly M, O'Loughlin C. The validity of the MacNew quality of life in heart disease questionnaire. Health Qual Life Outcomes 2004;2:6.
Contents lists available at ScienceDirect
Journal of Psychosomatic Research
The dimensional structure of the MacNew Health Related Quality of Life questionnaire: A Mokken Scale Analysis O. Friedrich a,⁎, J. Sipötz a,b, W. Benzer c, E. Kunschitz a,b, S. Höfer d a
Karl Landsteiner Institute for Scientific Research in Clinical Cardiology, Vienna, Austria Department of Cardiology, Hanusch Krankenhaus, Vienna, Austria Department of Interventional Cardiology, Academic Hospital, Feldkirch, Austria d Department of Medical Psychology, Innsbruck Medical University, Austria b c
a r t i c l e
i n f o
Article history: Received 9 February 2015 Received in revised form 16 April 2015 Accepted 17 April 2015 Keywords: MacNew Mokken Scale Analysis Health related quality of Life Coronary Artery Disease
a b s t r a c t Objective: The MacNew Health related Quality of Life Questionnaire is a widely used instrument for the assessment of health related quality of life in cardiac patients. The study addresses for the first time the dimensional structure of the MacNew with Mokken Scale Analysis (MSA). Methods: Separate exploratory MSA of the MacNew was conducted in a large Spanish (n = 1012) and a medium sized Austrian sample (n = 262) of patients with Coronary Artery Disease (CAD) after Percutaneous Coronary Intervention (PCI). The results of both samples were summarized in a synthesis model. Confirmatory MSA and Confirmatory Factor Analysis (CFA) were used to evaluate the model. Results: The synthesis model comprises 21 items forming a unidimensional sum scale of moderate strength. On the level of subdomains we define two strong unidimensional subscales (restriction: 6 items, and emotional: 10 items) and two smaller item sets (symptoms: 2 items and social: 3 items). 5 items were excluded due to low scalability in both samples. Conclusion: Our results generally support the use of the MacNew Global score, with the limitation, that five items may be questionable with regard to scalability. On the level of unidimensional subscales MSA suggests to differentiate between a six-item restriction scale and a ten-item emotional scale. The study demonstrates that Mokken Scale Analysis complements the results of factor analysis and can contribute to a more comprehensive understanding of the dimensional structure of Health-related Quality of Life questionnaires. © 2015 Elsevier Inc. All rights reserved.
Introduction The MacNew questionnaire is a widely used instrument for disease specific measurement of Health Related Quality of Life (HRQL) in cardiac patients. The questionnaire is originated in the Quality of Life after Myocardial Infarction (QLMI) interview, introduced by Oldridge et al. [1,2] in order to investigate the effects of cardiac rehabilitation after myocardial infarction in anxious and/or depressed patients. The QLMI instrument was generated by a clinimetric approach through interviews with physicians, nurses, health care professionals and patients with myocardial infarction leading to a set of 26 items covering five domains (symptoms, restrictions, confidence, self-esteem and emotional functioning). Based on the QLMI Lim et al. [3] and Valenti et al. [4] developed the patient self-administered MacNew questionnaire. The MacNew contains 27 questions, 24 items of the QLMI and three newly added items, ⁎ Corresponding author at: Karl Landsteiner Institute for Scientific Research in Clinical Cardiology, Hanusch Krankenhaus, Heinrich-Collin-Str. 30, A-1120 Vienna, Austria. Tel.: +43 1 680 212 3203; fax: +43 1 910 21 85219. E-mail address: [email protected] (O. Friedrich).
http://dx.doi.org/10.1016/j.jpsychores.2015.04.007 0022-3999/© 2015 Elsevier Inc. All rights reserved.
rated by a seven point Likert scale. The MacNew measures HRQL in three domains (emotional, physical, and social). A global scale can be calculated by summing up all items [4]. The MacNew has been made available in over 80 countries and validated in over 16 languages [5] in patients with a variety of cardiac conditions ranging from myocardial infarction to angina, acute coronary syndrome, heart failure and in patients with a pacemaker and is recommended as a core heart disease quality of life instrument [6]. The three-dimensional factor structure of the MacNew, originally proposed by Valenti et al. [4] for the Australian English version on the basis of Principal Component Analysis (PCA) is characterized by some ambiguity. About half of the items (12/26 — one item was excluded from analysis) showed substantial cross-loadings and were therefore allocated to more than one subscale. The three-factor solution was generally accepted for different language versions of the MacNew even if the results of factor analysis differed sometimes partly, especially in smaller samples and in particular with regard to items of the physical and social subscales [6]. In order to analyse HRQL on the level of the subdomains a unidimensional factor structure would be highly preferable. However, given the nature of HRQL (being multidimensional and
44
O. Friedrich et al. / Journal of Psychosomatic Research 79 (2015) 43–48
substantially interrelated) and the published results, the attempt to define such scales by allocating items according item loadings in factor analysis seems problematic. Our analysis aimed to readdress the factor structure of the German and Spanish MacNew questionnaire versions with a different approach by using Mokken Scale Analysis (MSA). MSA is based on the principles of Item Response Theory (IRT). It assumes that the probability of a given response is influenced by a latent trait (e.g. an individual's ability and item difficulty), and hence enables the evaluation of the scalability of single items with respect to this latent trait. Unlike parametric IRT techniques MSA does not require the assumption of a parametric form of item response functions [7,8]. MSA is particularly useful in order to investigate the dimensionality of scales. By employing a “bottom up” clustering search procedure using preselected cutoff points for item scalability, MSA allows, unlike other techniques, analysis of the dimensional structure of a scale on different hierarchical levels [9]. Our work comprises two parts. First, we conducted separate exploratory MSA of the MacNew in a large Spanish and a medium sized Austrian sample of patients with Coronary Artery Disease (CAD) undergoing Percutaneous Coronary Intervention (PCI). Second, we combined the results of both samples to a synthesis model, evaluated this model by confirmatory MSA and Confirmatory Factor Analysis (CFA) and compared it to existing models. Methods Participants
the dimensional structure on different hierarchical levels we employed a step by step approach starting with the established cutoff value for Hj (c = 0.3) and subsequently increasing by steps of 0.05 up to c = 0.6 [8]. Confirmatory MSA was conducted by calculating homogeneity coefficients of preselected item sets. Violations of monotonicity were assessed via CRIT statistic with values b 40 considered to be acceptable [11]. MSA was performed with R 3.0.3 and the R package Mokken [12,13]. The lower bound of scale reliability was estimated with Cronbach's α. Following recommendations with regard to Likert-type data we additionally calculated ordinal alpha, which is based on polychoric correlations instead of Pearson correlations and provides a more accurate estimate of reliability for our data [14]. Confirmatory Factor Analysis (CFA) CFA was executed in R 3.0.3 with the R package lavaan [15] and the graphical Structural Equation Modelling software Ωnyx [16]. Model fit was estimated by Comparative Fit Index (CFI), Tucker–Lewis Index (TLI) and root mean square error of approximation (RMSEA). CFI and TLI values N 0.90 indicate good fit, RMSEA b 0.08 suggest moderate, RMSEA b 0.06 good fit [17]. Because Mardia testing indicated nonnormal multivariate distribution we calculated robust indices with Satorra–Bentler correction [18]. Measurement errors of items within subscales were allowed to intercorrelate by referring to modification indices when appropriate. Results
The study sample comprised 1082 Spanish and 310 Austrian patients from the PRODES Xience Stent registry treated with PCI in 40 Spanish and 7 Austrian centres. The PRODES registry was designed to investigate changes in Health related Quality of Life and Mental Distress after PCI by a prospective, multi-centre approach. Patients with acute ST-elevation myocardial infarction (STEMI) or stent deployment during the last 6 months were excluded from the registry. The data were collected between January 1st, 2008 and December 31st, 2011. All Patients answered the MacNew questionnaire during hospital stay after PCI and agreed to participate by signing an informed consent. The registry was conducted in accordance to the Helsinki Declaration and the guidelines of Good Clinical Practice after approval by the ethical committee of the City of Vienna (EK-07-202-VK). Part of the Austrian data was previously published in another context elsewhere [10]. Mokken Scale Analysis (MSA) MSA assumes that the data fit three basic requirements constituting the Monotone Homogeneity Model (MHM): First, there are unidimensional latent traits capturing the association of item scores. Second, Item Response Curves are monotonically nondecreasing. Third, responses of one subject to the different items are stochastically independent [7]. If these assumptions are met item sum scores can be used to characterize and compare subjects with regard to a latent trait. MSA uses the Loevinger homogeneity coefficient Hj to describe the strength of the relationship between an item and the latent trait with high values suggesting a good ability to discriminate between low and high scores of the trait. Hj values ≥ 0.3 indicate scalability of an item. The total scale coefficient H reflects the discrimination power of sets of items constituting a scale. As a rule of thumb 0.3 ≤ H b 0.4 defines a weak scale, 0.4 ≤ H b 0.5 a moderate scale and H ≥ 0.5 a strong scale [7]. The scale search procedure in explanatory MSA follows a bottom-up approach. It starts off with the pair of items with the highest homogeneity coefficient Hj, and then stepwise adding items with maximizing the scale homogeneity coefficient H. The algorithm proceeds until no item exceeding a predefined cutoff value for Hj is left. Subsequently a new scale is formed following the same procedure. The process is finished when no items fulfilling the criterion of inclusion are left. To describe
Study sample The initial study samples comprised 1082 Spanish and 310 Austrian patients answering the MacNew questionnaire during hospital stay after PCI. The percentage of missing values of items 1 to 26 was between 0.9% and 2.6%. Item 27 (sexual intercourse) was skipped or answered with the “not applicable” response by 36% of participants. Because the MSA procedure requires complete datasets, we excluded all questionnaires with missing values for items 1–26. Item 27 was omitted from analysis. The final samples consisted of 1012 Spanish and 262 Austrian patients. Patient characteristics are summarized in Table 1. Exploratory Mokken Scale Analysis The results of the separate MSA for the Spanish and the Austrian sample are shown in Table 2. In general homogeneity coefficients were higher in the Austrian sample. With the cutoff value for Hj c = 0.3 in both samples a sumscale of moderate strength (Spain: H = 0.41, Austria: H = 0.44) was found. Item 3 of the Spanish sample and item 22 of the Austrian sample were unscalable (Hj b 0.3).
Table 1 Patient characteristics.
Age ± SD Male [%] Smoking [%] Hypercholesterolemia [%]a Hypertonia [%]a IDDM [%]a Obesity (BMI ≥ 30) [%] CCS classification [%]
Single vessel intervention [%] AHA classification [%]
CCS 0/I CCS II CCS III CCS IV AHA A AHA B AHA C
Spain (n = 1012)
Austria (n = 262)
62.7 ± 10.4 68.1 40.5 71.9 52.1 9.1 31.6 20.8 33.4 32.2 13.6 84.0 7.1 70.8 22.1
62.9 ± 9.4 71.4 33.9 87.5 70.2 7.9 26.7 15.1 52.3 23.4 9.2 81.0 9.7 62.6 27.7
IDDM: Insulin Dependent Diabetes Mellitus; CCS: Canadian Cardiovascular Society classification of angina pectoris; AHA American Heart Association classification of cardiac stenosis. a Treated.
O. Friedrich et al. / Journal of Psychosomatic Research 79 (2015) 43–48
45
Table 2 Separate explanatory MSA of the Spanish and the Austrian sample. Figures represent Loevinger item homogeneity coefficients (Hj) and scale homogeneity coefficients (H). Matching results highlighted in bold. S1: scale 1, S2: scale 2, S3: scale 3, S4: scale 4. Spain: N = 1012; Austria: N = 262. Item
Spain c = 0.3
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26
Frustrated Worthless Confident Down in the dumps Relaxed Worn out Happy Restless Short of breath Tearful More dependent Social activities Others/less confidence Chest pain Lack self-confidence Aching legs Sports/exercise limited Frightened Dizzy/lightheaded Restricted or limited Unsure about exercise Overprotective family Burden on others Excluded Unable to socialize Physically restricted H
Austria c = 0.50
c = 0.3
c = 0.55 S1
Sumscale
S1
S2
S3
S4
Sumscale
0.35 0.45 – 0.47 0.40 0.46 0.40 0.43 0.40 0.40 0.43 0.44 0.32 0.36 0.49 0.36 0.44 0.46 0.38 0.47 0.40 0.30 0.41 0.44 0.44 0.47 0.41
–
– 0.53 – 0.59 0.52 0.50 0.50 0.54
–
–
–
–
0.47 0.46 0.36 0.48 0.42 0.48 0.49 0.44 0.35 0.42 0.49 0.49 0.37 0.44 0.51 0.34 0.45 0.49 0.34 0.50 0.40 – 0.45 0.45 0.48 0.48 0.44
–
0.51 0.50 0.51 –
– 0.63 – 0.66 0.53 – 0.57 0.53 0.66 0.60
–
– 0.51
–
0.55 –
–
–
0.51 –
–
–
–
–
– 0.51
0.53
The scale search procedure revealed two strong subscales (H N 0.5) and two smaller item sets with the cutoff values for Hj c = 0.50 (Spain) and c = 0.55 (Austria): • In both samples scale 1 included six items of the physical/social domain (17, 20, 21 and 24–26). • Scale 2 consisted of items of the emotional domain: 9 items in the Spanish sample (2, 4–8, 10, 15, 18) and 8 items in the Austrian sample (1, 2, 4–8, 15). • Scale 3 in the Spanish sample comprised two items describing physical symptoms (9 and 14), which were excluded in the Austrian sample because of Hj b 0.55. With exception of the excluded item 23 (Hj b0.50), scale 4 of the Spanish sample corresponded to scale 3 of the Austrian sample, which combined items of the social domain (11, 12, 23). Scale 4 of the Austrian sample included two items of the emotional domain (10 and 18), which were part of scale 1 in the Spanish sample. • Five items (3, 13, 16, 19, 22) were excluded in both samples because of Hj b c at the respective levels.
Internal consistency was adequate for the sumscales and the subscales S1 and S2 in both samples (Cronbach's α ≥ 0.8, ordinal α ≥ 0.9). Lower reliability was found in the two smaller item sets S3 and S4 (Table 3). Synthesis model Combining the results of separate explanatory MSA at the cutoff values for Hj c = 0.50 (Spain) and c = 0.55 (Austria) we created a model proposal consisting of a restriction subscale with items covering restriction to physical or social activity connected to the heart problem, and an emotional subscale. In addition we defined two smaller sets (symptoms and social) including the items with higher discrimination power not assigned to either of the two subscales: • The restriction subscale includes the six physical/social items (17, 20, 21 and 24–26), which constituted scale 1 in both samples.
0.51
0.51
S2 0.64 0.61 – 0.64 0.55 0.55 0.60 0.58 –
–
–
– – – 0.64 – 0.67 0.59 – 0.57 0.59 0.66 0.62
S3
S4
–
–
–
– 0.57
– – 0.58 –
0.58 0.57 – –
– –
–
–
–
–
0.57 –
–
– 0.56
–
0.59
0.57
0.57
• The emotional subscale comprises ten items. Beside emotional items 2, 4–8, 15, which are part of scale 2 in both samples, we included item 1, which was missing in the Spanish sample because of Hj b c, and items 10 and 18, which constituted a own scale fragment in the Austrian sample, closely related to scale 2. • The symptom item set includes two items and corresponds to scale 3 in the Spanish sample with items 9 and 14 addressing symptoms of CAD (angina pectoris and dyspnoea). In the Austrian sample these items were not part of an item set at c = 0.55. • The social item set comprises items 11, 12, and 23 and corresponds to scale 3 of the Austrian sample and scale 4 of the Spanish sample with the single exception of item 12, which was missing in the Spanish sample because of Hj b 0.50. • The five items 3, 13, 16, 19, and 22 with lower discrimination power, not designated to item aggregates in either sample at c = 0.50 (Spain) and c = 0.55 (Austria), were excluded.
To ensure the created model fits the data of both samples we conducted confirmatory MSA for the Spanish and the Austrian sample separately. The results for the 4 subscales and the sumscale are summarized in Table 4. Homogeneity coefficients indicated high (H ≥ 0.5) discrimination power for the restriction and the emotional subscales and at least moderate discrimination power (H ≥ 0.4) for the two smaller sample sets in both samples. Because of excluding weaker items the moderate sumscale has gained strength (Spain: from 0.41 to 0.45; Austria: from 0.44 to 0.48). Reliability was high for the sumscale and both subscales (restriction and emotional), but not for the two smaller scale fragments (symptoms and social) (Table 5). Confirmatory Factor Analysis confirmed adequate fit of the sumscale and the 4-factor structure of synthesis model for the two samples separately and for both combined (Table 6). The total variance explained was 50.5% for the Spanish sample (restriction: 16.2%, emotional: 23.1%, symptoms: 4.7%, social: 6.6%), 53.3% for the Austrian sample (restriction: 16.6%, emotional: 25.0%, symptoms: 4.3%, social: 7.4%) and 50.8% for both samples combined (restriction: 16.2%, emotional: 23.3%, symptoms: 4.6%, social: 6.7%). Factor
Table 3 Reliability of scales resulting from explanatory MSA of the Spanish and the Austrian sample. Spain
Cronbach's α Ordinal α # items
Austria
Sumscale
S1
S2
S3
S4
Sumscale
S1
S2
S3
S4
0.94 0.95 25
0.88 0.91 6
0.90 0.91 9
0.66 0.68 2
0.64 0.72 2
0.94 0.96 25
0.89 0.92 6
0.91 0.92 8
0.76 0.82 3
0.66 0.72 2
46
O. Friedrich et al. / Journal of Psychosomatic Research 79 (2015) 43–48
Table 4 Separate confirmatory MSA of the synthesis model for the Austrian and the Spanish sample. Figures represent Loevinger item homogeneity coefficients (Hj) and scale homogeneity coefficients (H). Spain: N = 1012; Austria: N = 262. Item
Restriction Spain
1 2 4 5 6 7 8 9 10 11 12 14 15 17 18 20 21 23 24 25 26
Frustrated Worthless Down in the dumps Relaxed Worn out Happy Restless Short of breath Tearful More dependent Social activities Chest pain Lack self-confidence Sports/exercise limited Frightened Restricted or limited Unsure about exercise Burden on others Excluded Unable to socialize Physically restricted H
Emotional Austria
Symptoms
Spain
Austria
0.48 0.53 0.59 0.52 0.49 0.49 0.54
0.59 0.61 0.53 0.53 0.58 0.57 0.52
0.50
0.58
Spain
0.51
0.63
0.64
0.63 0.53
0.67 0.59
0.57 0.53 0.66 0.60
0.57 0.59 0.66 0.62
0.52
0.50
0.59
0.52
0.57
loadings and correlations between latent variables of the synthesis model are summarized in Fig. 1.
Discusssion The separate Mokken Scale Analysis of the Spanish and the Austrian sample led to very similar results and allowed for proposing a combined model of the unidimensional structure of the MacNew Health related Quality of Life Questionnaire. The model suggests two strong unidimensional subscales: The restriction subscale comprises all six items, which were designated to both, the physical and the social scale, in the original factor structure proposal by Valenti et al. [4]. These items are characterized by drawing a causal relationship between the term “heart problem” (“Herzproblem”, “problema de corazón”) and restrictions in physical or social activities (Items 17, 20, 24–26) or in case of item 21 addressing restriction in sports activity directly. The emotional subscale corresponds largely to the emotional subscale of the initially proposed three-factor model. It includes all eight items assigned exclusively to the emotional domain (1, 4–8, 10, 18) plus two items assigned to both the emotional and the social domain (2 and 15). In addition we defined two item sets comprising mainly items of the physical and the social subscale of the original factor structure proposal. Because of the small number of items (two and three, respectively), these aggregates showed lower levels of reliability and cannot legitimately be considered as fullfledged subscales. Nevertheless they may be of interest, since they contribute to a better understanding of the heterogeneity in the social and physical subscale. Two items exclusively designated to the physical subscale by Valenti et al. form an aggregate addressing the symptom status of CAD (angina pectoris and dyspnoea). Unlike the items of the restriction subscale these two questions are not specifically related to restriction of social or physical activities. Interestingly the two other
Spain
Sumscale Austria
0.48
0.51 0.54
Social Austria
0.51 0.47
0.58 0.57
0.47
0.56
0.48
0.57
0.48
0.51
0.48
Spain
Austria
0.37 0.47 0.49 0.42 0.47 0.42 0.45 0.41 0.41 0.44 0.46 0.37 0.51 0.45 0.47 0.50 0.41 0.42 0.46 0.46 0.49 0.45
0.48 0.48 0.49 0.43 0.49 0.51 0.46 0.35 0.44 0.51 0.51 0.45 0.53 0.48 0.51 0.53 0.43 0.46 0.48 0.50 0.52 0.48
exclusively physical items in the Valenti analysis (item 16: aching legs and item 19: feeling dizzy/lightheaded) were not part of this aggregate. This could possible be due to the fact, that both describe somatic symptoms, which are far less specific — at least with regard to CAD patients. The second item set includes three items covering dependence on others (items 11 and 23) and inability to meet social obligations in the usual way (item 12). In the original three-factor proposal these items were assigned to the social subscale exclusively (item 11), to all three subscales (item 12) and to the social and emotional subscale (item 23). They can be considered as a remainder of the social scale with items fitting neither the emotional nor the restriction subscale of our model. Five items (3, 13, 16, 19 and 22) were excluded because they showed low discrimination power and failed to build item sets in both samples. Overall our MSA is largely in line with the results of PCA reported by Valenti et al. [4] in essential respects. The two large blocks of items with similar loading patterns of this analysis – physical plus social restrictions and exclusively emotional – were assigned to two separate subscales. The differences between our results and the Valenti proposal can be largely explained by the different methodological approach. The MSA scale search procedure is specifically designed to define unidimensional scales in a hierarchical clustering process with evaluating items by their ability to discriminate with regard to a latent trait. Factor analysis methods in general and PCA in particular are, by contrast, primarily instruments for dimensionality reduction. They make no a priori assumptions about the dimensional structure of an item set and are therefore not particularly well suited to investigate the unidimensionality of item sets [9]. In neither of our both samples separate social and physical subscales were defined, because the scale search procedure first formed the restriction subscale, comprising items with loadings on the physical scale and the social scale, and thereafter the emotional
Table 5 Reliability of sumscale, subscales (restriction, emotional) and scale fragments (symptoms, social) of the synthesis model. Restriction
Cronbach's α Ordinal α # items
Emotional
Symptoms
Social
Sumscale
Spain
Austria
Spain
Austria
Spain
Austria
Spain
Austria
Spain
Austria
0.88 0.91 6
0.89 0.92 6
0.90 0.92 10
0.91 0.93 10
0.66 0.68 2
0.61 0.66 2
0.72 0.78 3
0.76 0.82 3
0.94 0.95 21
0.94 0.96 21
O. Friedrich et al. / Journal of Psychosomatic Research 79 (2015) 43–48
47
Table 6 Fit of synthesis model for the Spanish, the Austrian sample and both samples combined. Sumscalea
4 factors
CFI TLI RMSEA (CI) a
Spain
Austria
Combined
Spain
Austria
Combined
0.93 0.92 0.059 (0.056–0.063)
0.91 0.90 0.072 (0.064–0.079)
0.93 0.92 0.060 (0.057–0.063)
0.97 0.96 0.045 (0.040–0.049)
0.94 0.92 0.062 (0.054–0.071)
0.97 0.95 0.046 (0.043–0.050)
After allowing measurement errors to intercorrelate within subdomains when appropriate.
scale. Besides the small scale fragments, the then remaining items were too heterogeneous to form own domains, which is reflected by the variety of loading patterns of these items in the PCA of Valenti et al. The results of factor analysis and Mokken scaling are not contradictory but rather complement each other: While the loading patterns of factor analysis show how single items cover the three basic dimensions of Health-related Quality of Life (physical, emotional, social) [19], Mokken analysis allows defining separate subscales. The ambiguity of loading patterns in factor analysis of the MacNew reflects the fact that items covering Health-related Quality of Life are by nature not always exclusively assignable to one of the three basic domains of Quality of Life. Mokken analysis enables extracting unidimensional subscales, which can be used to compare distinct aspects of Health-related Quality of Life besides the level of the global score. In the literature on the dimensional structure of the MacNew questionnaire it is a common theme to point out that the model proposed by Valenti et al. is in need for revision, especially with regard to the physical and the social subscale. Critiques partly refer to divergent loading patterns found in validation studies in different language versions of the MacNew, which were also reported for the German [20–23] and the Spanish version [24]. However, such differences have to be evaluated with some caution because they may well be due, at least in part, to sample size issues and/or potential translation biases. The largest
study providing detailed results on a three dimensional structure of the MacNew confirmed the findings of Valenti et al. convincingly. Validating the Hebrew version in 773 patients undergoing CABG surgery, Geulayov et al. [25] recently reported an almost identical loading pattern in PCA with just one noteworthy variation — item 24 (excluded) without substantial cross-loading on the physical domain. We think the main concern with regard to results of factor analysis of the MacNew is not divergence in loading patterns but how to interpret these patterns given their inherent ambiguity. Two authors have proposed alternative models for the factor structure of the MacNew based on factor analysis. Gramm et al. [26] put forward a proposal with four unidimensional factors based on the results of a large German sample (n = 5692) of cardiac rehabilitation patients. He resolved the issue of ambiguous loading patterns in PCA by removing items with substantial cross-loadings in the three-factor structure and included only items with loadings of N 0.5 for one factor and b0.4 for all other factors in his analysis. Dempster et al. [27] proposed a fivefactor solution based on the results of a relatively small sample (n = 117) of patients with ischaemic heart disease from the UK. He achieved his unidimensional – with the exception of item 23, which was assigned to two factors – solution by grouping items with cross-loadings under the factor they loaded heaviest and/or designating items by reasons of content. Both proposals include an emotional subscale corresponding
Fig. 1. Summary of the synthesis model based on the Spanish and the Austrian data combined.
48
O. Friedrich et al. / Journal of Psychosomatic Research 79 (2015) 43–48
largely to the emotional scale of the original three-factor solution and our model. The remaining items were designated to three and four small factors, respectively, comprising between two and five items. We think that both of these proposals are problematic for methodological reasons. Gramm's strategy to omit items with cross-loadings is not based on item scalability and resulted in the exclusion of four of the six items constituting the restriction subscale of our model. Dempster's approach of assigning items according their strongest loading is also not convincing from a methodological point of view, especially given the relatively small sample size. Gramm, who tested this model on a large German sample, found, that it performed even worse than the threefactor model that he rejected in part in his paper. Strengths and limitations The strength of our study is that we for the first time applied an approach of probabilistic test theory to the MacNew. MSA not only allows evaluating the scalability of the items, but also is particularly suited to investigate the unidimensionality of the questionnaire. The study has also some limitations. First and foremost our analysis is confined to the psychometric evaluation of the MacNew exclusively. Further research is needed to prove the practical relevance of our results for clinical purposes, especially of the newly proposed restriction subscale. Second, our two samples comprised CAD patients after PCI only. Since the MacNew is recommended as a core heart disease quality of life instrument and used today in patients with a broad variety of cardiac conditions further research is needed to prove the generality of our results. Finally, a possible objection to our results may be, that our findings are biassed by the fact that participants answered the questionnaire during their hospital stay after PCI, especially with regard to items covering social interaction. To dispel that concern we have applied our model on six months follow up data of the Austrian sample and were able to confirm the findings of the baseline data with confirmatory MSA and CFA. Conclusion Summing up our results generally support the use of the MacNew Global score, with the limitation, that five of the 26 items (3, 13, 16, 19 and 22) may be questionable with regard to scalability. Further analysis including patients with various cardiac conditions has to prove if the latter is a general concern. This may easily be achieved by utilizing existing MacNew datasets in various languages. On the level of unidimensional subscales MSA suggests differentiating between a six-item restriction scale and a ten-item emotional scale. Overall the study demonstrates that in order to identify distinct unidimensional scales it is useful to combine the results of factor analysis with IRT-based methods like MSA. Conflict of interest declaration The authors declare no conflict of interest. Acknowledgements The work was supported by an unrestricted grant from Abbott Vascular (6095).
References [1] Oldridge N, Guyatt G, Jones N, et al. Effects on quality of life with comprehensive rehabilitation after acute myocardial infarction. Am J Cardiol 1991;67:1084–9. [2] Hillers TK, Guyatt GH, Oldridge N, et al. Quality of life after myocardial infarction. J Clin Epidemiol 1994;47:1287–96. [3] Lim LLY, Valenti LA, Knapp JC, et al. A self-administered quality-of-life questionnaire after acute myocardial infarction. J Clin Epidemiol 1993;46:1249–56. [4] Valenti L, Lim L, Heller RF, Knapp J. An improved questionnaire for assessing quality of life after acute myocardial infarction. Qual Life Res 1996;5:151–61. [5] macnew.org. MacNew health-related quality of life instrument. [Internet]. [cited 2015 Jan 1]; available from: http://www.macnew.org. [6] Höfer S, Lim L. The MacNew heart disease health-related quality of life instrument: a summary. Health Qual Life Outcomes 2004;2:10–1186. [7] Van Schuur WH. Mokken scale analysis: between the Guttman scale and parametric item response theory. Polit Anal 2003;11:139–63. [8] Stochl J, Jones PB, Croudace TJ. Mokken scale analysis of mental health and wellbeing questionnaire item responses: a non-parametric IRT method in empirical research for applied health researchers. BMC Med Res Methodol 2012;12:74. [9] Straat JH, van der Ark L, Sijtsma K. Methodological artifacts in dimensionality assessment of the hospital anxiety and depression scale (HADS). J Psychosom Res 2013; 74:116–21. [10] Sipötz J, Friedrich O, Höfer S, Benzer W, Chatsakos T, Gaul G. Health related quality of life and mental distress after PCI: restoring a state of equilibrium. Health Qual Life Outcomes 2013;11:144. [11] Molenaar IW, Sijtsma K. User's manual MSP5 for Windows. Groningen: IEC ProGAMMA; 2000. [12] van der Ark L. Mokken scale analysis in R. J Stat Softw 2007;20. [13] van der Ark L. New developments in Mokken scale analysis in R. J Stat Softw 2012;48. [14] Gadermann AM, Guhn M, Zumbo BD, Columbia B. Estimating ordinal reliability for Likert-type and ordinal item response data: a conceptual, empirical, and practical guide. Pract Assessment Res Eval 2012;17:1–12. [15] Rosseel Y. lavaan: an R package for structural equation modeling. J Stat Softw 2012;48. [16] Von Oertzen T, Brandmaier AM, Tsang S. Structural equation modeling with Ωnyx. Struct Equ Model A Multidiscip J 2014:1–14. [17] Hu L, Bentler PM. Cutoff criteria for fit indexes in covariance structure analysis: conventional criteria versus new alternatives. Struct Equ Model A Multidiscip J 1999;6: 1–55. [18] Curran P, West S, Finch J. The robustness of test statistics to nonnormality and specification error in confirmatory factor analysis. Psychol Methods 1996;1:16–29. [19] The WHOQOL Group. The World Health Organization quality of life assessment (WHOQOL): position paper from the World Health Organization. Soc Sci Med 1995;41:1403–9. [20] Höfer S, Benzer W, Brandt D, Laimer H, Schmid P, Bernardo A. MacNew Heart Disease Lebensqualitätsfragebogen nach Herzinfarkt: Die deutsche version. Z Klin Psychol Psychother 2004;33:270–80. [21] Höfer S, Anelli-Monti M, Berger T, Hintringer F, Oldridge N, Benzer W. Psychometric properties of an established heart disease specific health-related quality of life questionnaire for pacemaker patients. Qual Life Res 2005;14:1937–42. [22] Höfer S, Schmid J, Frick M. Psychometric properties of the MacNew heart disease health‐related quality of life instrument in patients with heart failure. J Eval Clin Pract 2008;14:500–6. [23] Höfer S, Saleem A, Stone J, Thomas R, Tulloch H, Oldridge N. The MacNew heart disease health-related quality of life questionnaire in patients with angina and patients with ischemic heart failure. Value Health 2012;15:143–50. [24] Cuixart Brotons C, Ribera Solé A, Permanyer Miralda G, et al. Adaptacion del cuestionario de calidad de vida postinfarto MacNew QLMI para su uso en la poblacion espanola. Med Clin 2000;115:768–71. [25] Geulayov G, Oldridge NB, Ziv A, Novokov I, Drory Y, Dankner R. Validation of the Hebrew version of the MacNew heart disease healthrelated quality of life questionnaire in patients undergoing coronary artery bypass surgery. J Clin Exp Cardiolog 2013;04. [26] Gramm L, Farin E, Jaeckel WH. Psychometric properties of the German version of the MacNew heart disease health-related quality of life questionnaire. Health Qual Life Outcomes 2012;10:83. [27] Dempster M, Donnelly M, O'Loughlin C. The validity of the MacNew quality of life in heart disease questionnaire. Health Qual Life Outcomes 2004;2:6.
