Knee Surg Sports Traumatol Arthrosc DOI 10.1007/s00167-014-3017-2
ANKLE
Validation of the Dutch language version of the Foot and Ankle Outcome Score I. N. Sierevelt • L. Beimers • C. J. A. van Bergen • D. Haverkamp • C. B. Terwee • G. M. M. J. Kerkhoffs
Received: 30 September 2013 / Accepted: 13 April 2014 Ó Springer-Verlag Berlin Heidelberg 2014
Abstract Purpose The aim of this study was to develop a Dutch language version of the Foot and Ankle Outcome Score (FAOS-DLV) and evaluate its measurement properties according to the definitions of the COnsensus-based Standards for the selection of health Measurement INstruments (COSMIN). Methods After a standard forward–backward translation procedure, the Dutch version of the FAOS was evaluated for reliability and validity in 110 patients with various hind foot and ankle complaints. Reliability was evaluated by calculation of intraclass correlation coefficients (ICC), Cronbach’s alpha for internal consistency, and the smallest detectable change (SDC). Construct validity of the FAOS was assessed by calculation of Spearman’s correlation coefficients with similar and dissimilar domains of the SF-36 health survey, American Orthopedic Foot and Ankle Society Ankle and Hindfoot Scale, and visual analogue scales for pain and disability. Dimensionality was tested with confirmatory factor analysis.
Results Reliability of the FAOS-DLV was good. The ICC of the subscales ranged from 0.83 to 0.88. The minimal value of Cronbach’s alpha was 0.76. The SDC at individual level ranged from 18 to 21 and at group level between 2.1 and 2.5. Construct validity was supported by confirmation of 85 % of the hypothesized correlations. Unidimensionality of the FAOS-DLV domains was moderate. Conclusion The Dutch version of the FAOS seems to have acceptable measurement properties. The questionnaire can be used for functional assessment of patients with varying hindfoot and ankle symptoms. It is, however, more suitable for clinical evaluation at group level than for monitoring a specific patient. Level of evidence Diagnostic study, Level I. Keywords Foot and Ankle Outcome Score (FAOS) Ankle PROM Validity Reliability Dutch translation
Introduction I. N. Sierevelt (&) D. Haverkamp Department of Orthopedics, Slotervaart Hospital, Louwesweg 6, 1066 EC Amsterdam, The Netherlands e-mail: [email protected] L. Beimers C. J. A. van Bergen G. M. M. J. Kerkhoffs Department of Orthopaedic Surgery, Orthopaedic Research Center Amsterdam, Academic Medical Center, University of Amsterdam, Meibergdreef 9, 1105 AZ Amsterdam, The Netherlands C. B. Terwee Department of Epidemiology and Biostatistics and the EMGO Institute for Health and Care Research, VU University Medical Center, van der Boechorststraat 7, 1081 BT Amsterdam, The Netherlands
Patient-reported outcome measures (PROMs) are of increasing importance in today’s clinical practice. For the evaluation of the effectiveness of treatment of conditions affecting the foot and ankle, numerous clinical scoring systems have been developed [2, 10, 16]. The choice for a specific scoring system is dependent on several factors, such as the type of patient, the outcome of interest, and the measurement properties of the instrument, such as its validity, reliability, and responsiveness to a change in health condition [16, 17]. Due to discrepancies on taxonomy, terminology, and definitions of measurement properties for health-related patient-reported outcomes (HRPRO), the COnsensus-based Standards for the selection of
123
Knee Surg Sports Traumatol Arthrosc
health Measurement INstruments (COSMIN) checklist was formulated [18, 19]. The checklist was developed to evaluate methodological quality per measurement property [26], but the system can also be used as guidance for the design and reporting of validation studies of HR-PROs and help create more uniformity. A widely used scoring system to evaluate symptoms and functional limitations related to a variety of foot and ankle conditions is the Foot and Ankle Outcome Score (FAOS) [21]. The patient self-administered FAOS is adapted from the Knee injury and Osteoarthritis Outcome Score (KOOS) [6, 22]. It has been validated in several languages, including English, Swedish, Turkish, and Persian [13, 20, 21], and for different foot and ankle pathologies, such as ankle ligament reconstruction [21], hallux valgus [3], and flat foot deformity [15]. Although there is increasing evidence to support the use of the FAOS, the score has not been adapted and validated for Dutch-speaking individuals. Together with the lack of Dutch validated instruments for functional assessment of patients with hindfoot and ankle conditions, there is a need for a validated Dutch language version of this questionnaire. The aim of this study was therefore to develop a Dutch language version of the Foot and Ankle Outcome Score (FAOS-DLV) and evaluate its measurement properties according to the COSMIN definitions.
Materials and methods Translation procedure The English version of the FAOS was translated into the Dutch language using the translation procedure according to the guidelines of cross-cultural adaptation from Guillemin et al. [11]. Forward translations were performed independently by two native English-speaking persons who were not related to the medical field. Subsequently, backward translation was performed by two independent native Dutch-speaking persons, one orthopaedic resident and one person with no medical background. Discrepancies were discussed and adjusted when necessary. Comprehensibility of the questionnaire was checked by a volunteer group of 15 patients with ankle and hindfoot complaints. None of the patients reported any problem in understanding the questions. Design A group of 110 randomly selected patients, who were native Dutch speaking, with ankle and hindfoot complaints visiting our outpatient clinic were asked to participate. After verbal informed consent, all patients received a first set of questionnaires containing the FAOS,
123
the Short Form 36 Health Survey (SF-36), visual analogue scales (VASs) for pain and disability, and were clinically evaluated at the outpatient clinic using the American Orthopedic Foot and Ankle Society (AOFAS) Ankle and Hindfoot Scale. The FAOS was re-administered 1 week later by mail, with an additional question to assess whether there were changes in complaints since the first assessment. To optimize the response rate, patients were contacted by phone to remind them to complete the second questionnaire. This number of patients was sufficient to yield adequate power for the main analysis. For additional confirmatory factor analysis, however, a sample size of 110 was not sufficient and was expanded to 250 patients. To lower the burden, only the first 110 patients received a full questionnaire and the remaining 140 were only asked to fill out the FAOS once. Outcome measures The outcome measures used in this study were the translated version of the Foot and Ankle Outcome Score (FAOS-DLV), a validated Dutch version of the SF-36, a visual analogue scale (VAS) for pain and disability, and the AOFAS Ankle and Hindfoot Scale. Foot and Ankle Outcome Score (FAOS) The FAOS is a self-administered questionnaire intended to evaluate symptoms and functional limitations related to the foot and ankle [21]. The questionnaire consists of 42 items covering five dimensions: pain, other symptoms, activities of daily living (ADL), sport and recreation, and foot- and ankle-related quality of life (QOL). Standardized answer options are given in five-point Likert-boxes where each answer option is scored from 0 to 4. Raw sum scores are transformed to a normalized score on a 0–100, extreme symptoms to no symptoms, scale. One or two missing answers are allowed substituted by the mean value of the subscale. The last week is taken into consideration when answering the questionnaire. It has been reported that the questionnaire takes 10 min to be completed [21]. SF-36 The SF-36 is a validated patient-administered, generic health-related quality of life (HRQL) instrument. It comprises 36 items across eight dimensions: physical functioning (PF), role-emotional functioning (RE), social functioning (SF), role physical functioning (RP), bodily pain (BP), vitality (VT), general health (GH), and mental health (MH) perceptions. Additional summary scores can be calculated from these domains for physical and mental health status (physical component score and mental
Knee Surg Sports Traumatol Arthrosc
component score). The eight dimensions of the SF-36 score are calculated on a 0–100, worst to best, scale [1, 28]. Visual analogue scale (VAS) Two 100-mm VASs were used to determine the severity of the ankle and foot pain (average pain over the last week) and disability (difficulty during daily activities over the past week). The VAS scores are presented on a 0–100, no pain or disability to most severe pain or disability, scale. American Orthopedic Foot and Ankle Society (AOFAS) Ankle and Hindfoot Scale The AOFAS Ankle Hindfoot Scale is one of the four rating systems developed by the AOFAS to provide a standard method of reporting the clinical status of the ankle and foot [14]. The AOFAS Ankle and Hindfoot Scale incorporates both patient-based and physical exam items, and converts the answer options into numerical scales to describe pain function and alignment. Scores are calculated on a 0–100, worst to best, scale. Forty points are assigned to pain, 50 to function, and 10 to alignment [14]. Statistical analysis Measurement properties such as reliability, construct validity, and interpretability were assessed as described below. Statistical analysis was performed by use of IBM SPSS 20.0 (SPSS Inc. Chicago, IL). Reliability Reliability of a questionnaire is the extent to which the measurement is free from measurement error [19]. Reliability in this study was assessed by internal consistency, test–retest reliability, and measurement error. Internal consistency is the degree of interrelatedness among the items of (sub) scales of the questionnaire [19]. Cronbach’s alpha was calculated to evaluate the internal consistency of the Dutch FAOS [4]. A Cronbach’s alpha between 0.7 and 0.95 was considered as satisfactory [25]. Additional unidimensionality of the subscales was checked using confirmatory factor analyses (CFA), as described in the section ‘Construct validity’. Test–retest reliability is the ability to measure the same way twice in patients with unchanged complains [19]. An additional question was added to the second questionnaire to assess whether the patient’s situation was stable. Patients who reported change in complaints were excluded from the test–retest analysis. Test–retest reliability was assessed by calculation of the intra class coefficients (ICCagreement,
two-way random-effects model) [8]. An ICC [ 0.7 was considered as good [24, 29]. Measurement error is the systematic and random error of a patient’s score that is not attributed to true changes in the construct(s) to be measured [19]. The standard error of measurement (SEM) was calculated as the square root of the within-subject variance (i.e. sum of the between-measures variance and the residual variance) [5]. Additionally, the smallest detectable change (SDC) was calculated from the SEM at individual level (1.96*H2*SEM) and at group level (SDC/Hn) [9]. Construct validity Validity is the degree to which the instrument measures the construct(s) it intends to measure [19]. In the absence of a gold standard, validity was assessed in terms of construct validity, which is the extent to which a score relates to other scores consistent with theoretically derived hypotheses [19]. Construct validity was evaluated by analysing the association between the FAOS subscales and the SF-36 domains, the AOFAS and the VASs by use of Spearman’s correlation coefficients. A coefficient between 0.4 and 0.7 (or -0.4 and -0.7) was defined as a moderate correlation, and coefficients lower than 0.4 (or higher than -0.4) were considered unrelated or measuring different constructs [3]. A priori hypotheses were formulated to evaluate construct validity; moderate correlations were expected between all five FAOS subscales with the physical SF-36 domain scores (PF, RP, BP), the physical component score (PCS), the AOFAS, and the VASs, and low correlations (\0.4) with the mental SF-36 domain scores (VT, RE, MH) and the mental component score (MCS). Confirmation of at least 75 % of the a priori hypotheses indicates sufficient construct validity [25]. Additional structural validity was assessed. This is the degree to which the scores of a HR-PRO instrument are an adequate reflection of the dimensionality of the construct to be measured [19]. For this purpose, CFA was performed using the FAOS data of all 250 participants. Model fit was evaluated by comparing the root mean square error of approximation (RMSEA), Comparative Fit Index (CFI), and Tucker–Lewis Fit Index (TLI). An RMSEA B0.06 indicated good model fit, and for CFI and TLI, a cut-off value of 0.95 was chosen [12, 23]. Interpretability Interpretability is the degree to which one can assign a qualitative meaning on a quantitative score or change in score [19]. One aspect of interpretability is the presence of floor and ceiling effects. If many patients score at the
123
Knee Surg Sports Traumatol Arthrosc
boundaries (worst and best possible score) of a questionnaire, the questionnaire can fail to demonstrate a decrease in patients demonstrating signs of clinical deterioration and vice versa. These floor and ceiling effects will limit its responsiveness and reliability. In this study, floor or ceiling effects were defined to be present if more than 15 % of the patients reported the worst or the best possible score [25].
missing from 83 questionnaires, which implies 1.4 % of all items, ranging from 0 (QOL) to 3.4 % (Sport) per domain. Calculation of sub scores could therefore not be performed for four domain scores (3 Sport and 1 Quality of Life) at baseline and two domain scores (all from the Sport domain) at retest. Reliability
Results One hundred and ten patients participated in the study. The population consisted of 59 (54 %) males and 51 (46 %) females with a mean age of 40 years (SD 13.6). Median duration of complaints was 36 (range 6–240) months, and 64 patients had undergone an operation, of which 20 cases were open and 44 involved arthroscopic surgery. Patients were diagnosed with hindfoot and ankle osteoarthritis (22 %), impingement (19 %), osteochondral defect or loose body (25 %), instability (4 %), tendinopathy (5 %), or other (25 %). A total of 83 (75 %) patients returned their second questionnaire, 8 patients reported a change in complaints and were not included in the test–retest reliability analysis. The FAOS scores of the first assessment are presented in Table 1. Missing data At baseline, 79 (1.7 %) from a total of 4,620 FAOS items (110 patients 9 42 items) were missing, ranging from 0.5 (Symptoms) to 4.0 % (Sport). At retest, 48 items were
Table 1 Descriptives and percentages highest and lowest scores of the FAOS, SF-36 domains, AOFAS, and VASs
PF physical function, RP role physical, BP bodily pain, GH general health, MCS mental component summary score, VT vitality, SF social function, RE role-emotional, and MH mental health, PCS physical component summary score, MCS mental component score
123
N = 110
Mean (SD)
FAOS pain
58.1 (19.9)
FAOS symptoms FAOS ADL
Cronbach’s alpha, intraclass correlation coefficients (ICC), SEM, and SDC are presented in Table 2. Unidimensionality of the domains was moderate, as described in the section ‘Construct validity’. The Cronbach’s alpha of the five subscales of the FAOS ranged from 0.76 (Symptoms) to 0.95 (ADL), which indicates a good internal consistency. For all subscales, the ICCs were between 0.83 (ADL) and Table 2 Reliability results of the FAOS FAOS pain
FAOS symptoms
FAOS ADL
FAOS sport/ Rec
FAOS QOL
Crohnbach’s alpha (n = 110)
0.91
0.76
0.95
0.87
0.77
ICC (n = 75)
0.85
0.88
0.83
0.87
0.86
SEM (n = 75)
6.5
6.6
7.7
7.7
6.5
SDC individual (n = 75)
18.0
18.4
21.4
21.5
18.2
2.1
2.1
2.5
2.5
2.1
SDC group (n = 75)
ICC intraclass correlation coefficient, SEM standard error of measurement, SDC smallest detectable change
Median (range)
Highest score (%)
Lowest score (%)
56 (6–100)
0
2 (1.8 %)
54.1 (21.1)
54 (7–96)
0
0
71.1 (19.8)
72 (17–100)
0
5 (4.5 %)
FAOS sport/Rec
37.4 (23.2)
35 (0–100)
1 (0.9 %)
2 (1.9 %)
FAOS QOL
30.0 (19.2)
31 (0–88)
7 (6.4 %)
0
SF-36 PF
57.5 (21.5)
55 (5–95)
0
0
SF-36 RP
45.9 (40.2)
50 (0–100)
34 (30.6 %)
31 (27.9 %)
SF-36 BP
46.5 (18.8)
41 (0–100)
3 (2.7 %)
1 (0.9 %)
SF-36 GH
69.6 (19.9)
72 (10–100)
0
7 (6.4 %)
SF-36 VT
61.1 (19.2)
60 (5–100)
0
3 (2.7 %)
SF-36 SF
73.5 (24.2)
75 (0–100)
2 (1.8 %)
31 (27.9 %)
SF-36 RE
77.8 (33.8)
100 (0–100)
11 (9.9 %)
68 (61.3 %)
SF-36 MH
75.3 (19.8)
76 (0–100)
1 (0.9 %)
17 (15.3 %)
VAS pain
51.5 (23.1)
55 (0–98)
1 (0.9)
0
VAS disability
53.8 (25.9)
60 (0–99)
2 (1.8 %)
0
AOFAS
57.0 (19.7)
60 (14–100)
0
1 (0.9 %)
Knee Surg Sports Traumatol Arthrosc Table 3 Spearman’s correlation coefficients (r) between FAOS scores and SF-36 subscale scores, AOFAS, and VASs (n = 110) FAOS pain
FAOS symptoms
FAOS ADL
FAOS sport/Rec
FAOS QOL
SF36-PF
0.62
0.47
0.67
0.63
0.54
SF36-RP
0.38
0.30
0.42
0.28
0.33
SF36-BP SF36-GH
0.60 0.18
0.36 0.07
0.57 0.21
0.57 0.09
0.60 0.22
SF36-VT
0.21
0.16
0.25
0.18
0.31
SF36-SF
0.39
0.37
0.49
0.40
0.49
SF36-RE
0.21
0.32
0.21
0.09
0.15
SF36-MH
0.35
0.24
0.35
0.24
0.38
SF36-PCS
0.55
0.32
0.59
0.54
0.53
SF36-MCS
0.22
0.27
0.25
0.15
0.26
AOFAS
0.56
0.33
0.44
0.47
0.47
AOFAS pain
0.55
0.33
0.44
0.41
0.39
AOFAS function
0.49
0.30
0.40
0.43
0.45
VAS pain
-0.66
-0.46
-0.53
-0.46
-0.63
VAS disability
-0.55
-0.38
-0.48
-0.48
-0.66
PF physical function, RP role physical, BP bodily pain, GH general health, MCS mental component summary score, VT vitality, SF social function, RE role-emotional, and MH mental health, PCS physical component summary score, MCS mental component score
0.88 (Symptoms). Test–retest reliability was therefore good. At the individual level, the SDC ranged from 18.0 (Pain) to 21.5 (Sport/Rec) and at group level from 2.1 (Pain, Symptoms, and Qol) to 2.5 (ADL and Sport/Rec) for the FAOS subscales (Table 2). Construct validity As hypothesized, a moderate correlation was observed between 4 FAOS subscales ‘Pain’, ‘ADL’, ‘Sport/Recreation’, and ‘QOL’ and the physical domains of the SF-36 (PF, BP), the PCS, the AOFAS, and both VASs (r [ 0.4 or r \ -0.4). However, correlations with the FAOS subscale ‘Symptoms’ and with the SF-36 domain Role Physical were generally lower than hypothesized (Table 3). The poor correlations between the 3 FAOS subscales for pain, ADL, and sport/recreation and the SF-36 domains concerning mental health (VT, RE, MH) and the MCS were confirmed (r \ 0.4). A total of 85 % of all a priori hypotheses was confirmed. CFA demonstrated a RMSEA of 0.09, a CFI of 0.92, and a TLI of 0.92, which indicated moderate model fit of the five-factor structure. Correlations among the five subscales ranged from 0.6 (Symptoms-ADL) to 0.8 (Pain-ADL).
Ceiling and floor effects For none of the individual FAOS-DLV domains, the percentage worst (0) or best (100) possible score had exceeded the predefined 15 %. Thus, no floor or ceiling effects were detected for any of the FAOS domains. However, clear floor and ceiling effects were observed for the SF-36 domains role physical, social function, role-emotional and mental health (Table 1).
Discussion The main results in this study revealed that both reliability and validity of the Dutch version of the FAOS were sufficient. The measurement properties of this Dutch language version of the FAOS were investigated in a population with varying ankle and hindfoot conditions. Good measurement properties of the questionnaire are warranted for precise measurement and evaluation of an intervention. To avoid confusing terminology concerning these measurement properties, the COSMIN checklist was used for reporting this study. As reported in other validation studies of the FAOS, only very few items were missing (1.6 % at baseline and 1.4 % at retest) [20, 21]. The subscale ‘Sport/Rec’ presented the most missing values (4 %). Nevertheless, scores of all subscales could be calculated. The internal consistency of all the subscales of the questionnaire was good. Cronbach’s alpha varied between 0.76 (Symptoms) and 0.95 (ADL) and was comparable to the results of other validation studies of the FAOS [20, 21]. The test–retest reliability (ICC) was high for all subscales of the FAOS, as was reported in all other validation studies of the FAOS [3, 13, 15, 20, 21]. Despite the good test–retest reliability, the SDC of all subscales was large and ranged between 18 and 21 points at individual level. This could be explained by the fact that the ICC is strongly influenced by the variability between the patients [9]. A heterogeneous population, as was included in our study, will yield a higher ICC than a homogeneous population. However, the SEMs found in our study (and therefore the SDC) were comparable to the SEMs reported in other validation studies of HR-PROs [6, 7]. The SEM is independent of the variability between the patients and is an important property for evaluative purposes in clinical settings [9]. The resulting SDC is the minimal change score that can be attributed to true change and not to measurement error. For group comparisons with our sample size of 75 patients, this value ranged between the 2.1 and 2.5 points. In case of monitoring a single patient overtime, an increase or decrease of at least 18–21.5 points has to be achieved to be interpreted as real change.
123
Knee Surg Sports Traumatol Arthrosc
Negabhan et al. [20] reported smaller SDCs in their validation study. However, they used another formula to calculate the SDC that could underestimate the value of the SDC. Terwee et al. [27] reported that it is likely that many PROMs are not suitable for evaluation at individual level due to large SEMs and are therefore not able to detect minimal but clinically important changes. However, at group level, the SDC is smaller, and with the right sample sizes, clinically important changes can be detected [27]. For proper reliability assessment, the patient should fill out the questionnaire twice under the same circumstances [18]. However, this criterion was not met in our study since the participants filled out the first questionnaire at the outpatient clinic and the second one at home. This could also attribute to the large SDC in our study. Despite some limited measurement properties, the SF36, AOFAS, and the VASs were the most appropriate instruments for the assessment of construct validity of the FAOS. They were available in Dutch language version, suitable for the study population, and enabled comparisons with other validation studies. Construct validity was sufficient considering the hypothesized moderate correlations between most of the subscales of the FAOS and the scales measuring similar concepts and poor correlations with those measuring dissimilar concepts. These results are comparable to the results of other validation studies [20, 21]. However, the absolute values of the correlation coefficients between the physical outcomes and the subscale ‘Symptoms’ were generally smaller than expected (\0.4). These results are comparable to those of Negahban et al. [20] and Chen et al. [3], who also reported low correlations between the SF36 domains and this subscale. Chen et al. [3] attributed these poor correlations to the foot-specific nature of the questions used in this subscale. In contrast, Karathepe et al. [13] presented correlations with the SF-36 domains BP, PF, and SF of 0.71, 0.73, and 0.53, respectively. It is possible that our hypothesis that this subscale was moderately correlated to other physical scales was too optimistic, since the subscale ‘Symptoms’ does comprise items such as swelling and crepitation, aspects that do not necessarily have to be related to physical function or pain. However, considering our small values of the correlation coefficients, the subscale ‘Symptoms’ was still stronger associated with physical outcome measures than with the domain scores concerning mental health. Additionally, model fit of the five-factor structure was only moderate, which could be explained by the high correlations between the subscales within the questionnaire itself. More research is needed to make proper conclusion on the unidimensionality of the constructs. Similar to other studies [13, 20], the percentage of patients having the worst or best possible score did not
123
exceed the predefined 15 %, demonstrating the absence of floor and ceiling effects for all subscales of the FAOS. However, Roos et al. [21] reported substantial ceiling effects for all subscales of the FAOS, all exceeding 15 %, which they attributed to the long period of 12 years after ligament reconstruction of the ankle. The main limitation of this study is that pre- and postinterventional assessments were not incorporated in the study design. As a consequence, neither the responsiveness nor the minimal important change in the FAOS-DLV could be assessed. Information about these measurement properties is required to be able to evaluate the appropriateness for evaluative trials and to calculate sample sizes, and should be assessed in additional studies. The clinical relevance of this validation study is that it confirms that the Dutch version of the FAOS is valid and comparable to the available versions in other languages, making the questionnaire appropriate for clinical trials and other evaluation purposes. The obtained results demonstrate that the questionnaire can be used for patients with a variety of ankle and hindfoot conditions.
Conclusion The Dutch language version of the FAOS (FAOS-DLV) seems to have adequate measurement properties for use in patients with varying ankle and hindfoot conditions. The questionnaire is, however, more suitable for clinical evaluation at group level than for monitoring a specific patient. Further research can clarify the unidimensionality of the subscales of the FAOS-DLV. Acknowledgments The authors would like to thank Marlijn de Beer for her outstanding work in patient recruitment and data management, and John D. King for reading and editing the manuscript. Conflict of interest of interest.
The authors declare that they have no conflict
References 1. Aaronson NK, Muller M, Cohen PD, Essink-Bot ML, Fekkes M, Sanderman R, Sprangers MA, Te Velde A, Verrips E (1998) Translation, validation, and norming of the Dutch language version of the SF-36 Health Survey in community and chronic disease populations. J Clin Epidemiol 51:1055–1068 2. Button G, Pinney S (2004) A meta-analysis of outcome rating scales in foot and ankle surgery: is there a valid, reliable, and responsive system? Foot Ankle Int 25:521–525 3. Chen L, Lyman S, Do H, Karlsson J, Adam SP, Young E, Deland JT, Ellis SJ (2012) Validation of Foot and Ankle Outcome Score for hallux valgus. Foot Ankle Int 33:1145–1155 4. Cronbach LJ (1951) Coefficient alpha and the internal structure of tests. Psychometrika 16:297–334
Knee Surg Sports Traumatol Arthrosc 5. De Boer MR, de Vet HCW, Terwee CB, Moll AC, Vo¨lker-Dieben HJM, van Rens GHMB (2005) Changes to the subscales of two vision-related quality of life questionnaires are proposed. J Clin Epidemiol 58:1260–1268 6. De Groot IB, Favejee MM, Reijman M, Verhaar JA, Terwee CB (2008) The Dutch version of the knee injury and osteoarthritis outcome score: a validation study. Health Qual Life Outcomes 6:16 7. De Groot IB, Reijman M, Terwee CB, Bierma-Zeinstra SMA, Favejee M, Roos EM, Verhaar JAN (2007) Validation of the Dutch version of the Hip disability and Osteoarthritis Outcome Score. Osteoarthr Cartil 15:104–109 8. De Vet HCW, Terwee CB, Knol DL, Bouter LM (2006) When to use agreement versus reliability measures. J Clin Epidemiol 59:1033–1039 9. De Vet HC, Terwee CB, Ostelo RW, Beckerman H, Knol DL, Bouter LM (2006) Minimal changes in health status questionnaires: distinction between minimally detectable change and minimally important change. Health Qual Life Outcomes 4:54 10. Eechaute C, Vaes P, Van Aerschot L, Asman S, Duquet W (2007) The clinimetric qualities of patient-assessed instruments for measuring chronic ankle instability: a systematic review. BMC Musculoskelet Disord 8:6 11. Guillemin F, Bombardier C, Beaton D (1993) Cross-cultural adaptation of health-related quality of life measures: literature review and proposed guidelines. J Clin Epidemiol 46:1417–1432 12. Hu L, Bentler P (1999) Cutoff criteria for fit indexes in covariance structure analysis: conventional criteria versus new alternatives. Struct Equ Model 6:1–55 ¨ zbek G (2009) 13. Karatepe AG, Gu¨naydin R, Kaya T, Karlibas¸ U, O Validation of the Turkish version of the foot and ankle outcome score. Rheumatol Int 30:169–173 14. Kitaoka HB, Alexander IJ, Adelaar RS, Nunley JA, Myerson MS, Sanders M (1994) Clinical rating systems for the ankle-hindfoot, midfoot, hallux, and lesser toes. Foot Ankle Int 15:349–353 15. Mani SB, Brown HC, Nair P, Chen L, Do HT, Lyman S, Deland JT, Ellis SJ (2013) Validation of the Foot and Ankle Outcome Score in Adult Acquired Flatfoot Deformity. Foot Ankle Int 34:1140–1146 16. Martin RL, Irrgang JJ (2007) A survey of self-reported outcome instruments for the foot and ankle. J Orthop Sports Phys Ther 37:72–84 17. Martin RL, Irrgang JJ, Lalonde KA, Conti S (2006) Current concepts review: foot and ankle outcome instruments. Foot Ankle Int 27:383–390
18. Mokkink LB, Terwee CB, Patrick DL, Alonso J, Stratford PW, Knol DL, Bouter LM, De Vet HCW (2010) The COSMIN checklist for assessing the methodological quality of studies on measurement properties of health status measurement instruments: an international Delphi study. Qual Life Res 19:539–549 19. Mokkink LB, Terwee CB, Patrick DL, Alonso J, Stratford PW, Knol DL, Bouter LM, de Vet HCW (2010) The COSMIN study reached international consensus on taxonomy, terminology, and definitions of measurement properties for health-related patientreported outcomes. J Clin Epidemiol 63:737–745 20. Negahban H, Mazaheri M, Salavati M, Sohani SM, Askari M, Fanian H, Parnianpour M (2010) Reliability and validity of the foot and ankle outcome score: a validation study from Iran. Clin Rheumatol 29:479–486 21. Roos EM, Brandsson S, Karlsson J (2001) Validation of the foot and ankle outcome score for ankle ligament reconstruction. Foot Ankle Int 22:788–794 22. Roos EM, Roos HP, Ekdahl C, Lohmander LS (1998) Knee injury and Osteoarthritis Outcome Score (KOOS)—validation of a Swedish version. Scand J Med Sci Sports 8:439–448 23. Schreiber JB, Nora A, Stage FK, Barlow EA, King J (2006) Reporting structural equation modeling and confirmatory factor analysis results: a Review. J Educ Res 99:323–337 24. Streiner DL, Norman GR (2008) Health measurement scales: a practical guide to their development and use. In: 2nd (ed) Oxford University Press, New York 25. Terwee CB, Bot SDM, de Boer MR, van der Windt DAWM, Knol DL, Dekker J, Bouter LM, de Vet HCW (2007) Quality criteria were proposed for measurement properties of health status questionnaires. J Clin Epidemiol 60:34–42 26. Terwee CB, Mokkink LB, Knol DL, Ostelo RWJG, Bouter LM, de Vet HCW (2012) Rating the methodological quality in systematic reviews of studies on measurement properties: a scoring system for the COSMIN checklist. Qual Life Res 21:651–657 27. Terwee CB, Roorda LD, Knol DL, De Boer MR, De Vet HCW (2009) Linking measurement error to minimal important change of patient-reported outcomes. J Clin Epidemiol 62:1062–1067 28. Ware JE, Sherbourne CD (1992) The MOS 36-item short-form health survey (SF-36). I. Conceptual framework and item selection. Med Care 30:473–483 29. Weir JP (2005) Quantifying test-retest reliability using the intraclass correlation coefficient and the SEM. J Strength Cond Res 19:231–240
123
ANKLE
Validation of the Dutch language version of the Foot and Ankle Outcome Score I. N. Sierevelt • L. Beimers • C. J. A. van Bergen • D. Haverkamp • C. B. Terwee • G. M. M. J. Kerkhoffs
Received: 30 September 2013 / Accepted: 13 April 2014 Ó Springer-Verlag Berlin Heidelberg 2014
Abstract Purpose The aim of this study was to develop a Dutch language version of the Foot and Ankle Outcome Score (FAOS-DLV) and evaluate its measurement properties according to the definitions of the COnsensus-based Standards for the selection of health Measurement INstruments (COSMIN). Methods After a standard forward–backward translation procedure, the Dutch version of the FAOS was evaluated for reliability and validity in 110 patients with various hind foot and ankle complaints. Reliability was evaluated by calculation of intraclass correlation coefficients (ICC), Cronbach’s alpha for internal consistency, and the smallest detectable change (SDC). Construct validity of the FAOS was assessed by calculation of Spearman’s correlation coefficients with similar and dissimilar domains of the SF-36 health survey, American Orthopedic Foot and Ankle Society Ankle and Hindfoot Scale, and visual analogue scales for pain and disability. Dimensionality was tested with confirmatory factor analysis.
Results Reliability of the FAOS-DLV was good. The ICC of the subscales ranged from 0.83 to 0.88. The minimal value of Cronbach’s alpha was 0.76. The SDC at individual level ranged from 18 to 21 and at group level between 2.1 and 2.5. Construct validity was supported by confirmation of 85 % of the hypothesized correlations. Unidimensionality of the FAOS-DLV domains was moderate. Conclusion The Dutch version of the FAOS seems to have acceptable measurement properties. The questionnaire can be used for functional assessment of patients with varying hindfoot and ankle symptoms. It is, however, more suitable for clinical evaluation at group level than for monitoring a specific patient. Level of evidence Diagnostic study, Level I. Keywords Foot and Ankle Outcome Score (FAOS) Ankle PROM Validity Reliability Dutch translation
Introduction I. N. Sierevelt (&) D. Haverkamp Department of Orthopedics, Slotervaart Hospital, Louwesweg 6, 1066 EC Amsterdam, The Netherlands e-mail: [email protected] L. Beimers C. J. A. van Bergen G. M. M. J. Kerkhoffs Department of Orthopaedic Surgery, Orthopaedic Research Center Amsterdam, Academic Medical Center, University of Amsterdam, Meibergdreef 9, 1105 AZ Amsterdam, The Netherlands C. B. Terwee Department of Epidemiology and Biostatistics and the EMGO Institute for Health and Care Research, VU University Medical Center, van der Boechorststraat 7, 1081 BT Amsterdam, The Netherlands
Patient-reported outcome measures (PROMs) are of increasing importance in today’s clinical practice. For the evaluation of the effectiveness of treatment of conditions affecting the foot and ankle, numerous clinical scoring systems have been developed [2, 10, 16]. The choice for a specific scoring system is dependent on several factors, such as the type of patient, the outcome of interest, and the measurement properties of the instrument, such as its validity, reliability, and responsiveness to a change in health condition [16, 17]. Due to discrepancies on taxonomy, terminology, and definitions of measurement properties for health-related patient-reported outcomes (HRPRO), the COnsensus-based Standards for the selection of
123
Knee Surg Sports Traumatol Arthrosc
health Measurement INstruments (COSMIN) checklist was formulated [18, 19]. The checklist was developed to evaluate methodological quality per measurement property [26], but the system can also be used as guidance for the design and reporting of validation studies of HR-PROs and help create more uniformity. A widely used scoring system to evaluate symptoms and functional limitations related to a variety of foot and ankle conditions is the Foot and Ankle Outcome Score (FAOS) [21]. The patient self-administered FAOS is adapted from the Knee injury and Osteoarthritis Outcome Score (KOOS) [6, 22]. It has been validated in several languages, including English, Swedish, Turkish, and Persian [13, 20, 21], and for different foot and ankle pathologies, such as ankle ligament reconstruction [21], hallux valgus [3], and flat foot deformity [15]. Although there is increasing evidence to support the use of the FAOS, the score has not been adapted and validated for Dutch-speaking individuals. Together with the lack of Dutch validated instruments for functional assessment of patients with hindfoot and ankle conditions, there is a need for a validated Dutch language version of this questionnaire. The aim of this study was therefore to develop a Dutch language version of the Foot and Ankle Outcome Score (FAOS-DLV) and evaluate its measurement properties according to the COSMIN definitions.
Materials and methods Translation procedure The English version of the FAOS was translated into the Dutch language using the translation procedure according to the guidelines of cross-cultural adaptation from Guillemin et al. [11]. Forward translations were performed independently by two native English-speaking persons who were not related to the medical field. Subsequently, backward translation was performed by two independent native Dutch-speaking persons, one orthopaedic resident and one person with no medical background. Discrepancies were discussed and adjusted when necessary. Comprehensibility of the questionnaire was checked by a volunteer group of 15 patients with ankle and hindfoot complaints. None of the patients reported any problem in understanding the questions. Design A group of 110 randomly selected patients, who were native Dutch speaking, with ankle and hindfoot complaints visiting our outpatient clinic were asked to participate. After verbal informed consent, all patients received a first set of questionnaires containing the FAOS,
123
the Short Form 36 Health Survey (SF-36), visual analogue scales (VASs) for pain and disability, and were clinically evaluated at the outpatient clinic using the American Orthopedic Foot and Ankle Society (AOFAS) Ankle and Hindfoot Scale. The FAOS was re-administered 1 week later by mail, with an additional question to assess whether there were changes in complaints since the first assessment. To optimize the response rate, patients were contacted by phone to remind them to complete the second questionnaire. This number of patients was sufficient to yield adequate power for the main analysis. For additional confirmatory factor analysis, however, a sample size of 110 was not sufficient and was expanded to 250 patients. To lower the burden, only the first 110 patients received a full questionnaire and the remaining 140 were only asked to fill out the FAOS once. Outcome measures The outcome measures used in this study were the translated version of the Foot and Ankle Outcome Score (FAOS-DLV), a validated Dutch version of the SF-36, a visual analogue scale (VAS) for pain and disability, and the AOFAS Ankle and Hindfoot Scale. Foot and Ankle Outcome Score (FAOS) The FAOS is a self-administered questionnaire intended to evaluate symptoms and functional limitations related to the foot and ankle [21]. The questionnaire consists of 42 items covering five dimensions: pain, other symptoms, activities of daily living (ADL), sport and recreation, and foot- and ankle-related quality of life (QOL). Standardized answer options are given in five-point Likert-boxes where each answer option is scored from 0 to 4. Raw sum scores are transformed to a normalized score on a 0–100, extreme symptoms to no symptoms, scale. One or two missing answers are allowed substituted by the mean value of the subscale. The last week is taken into consideration when answering the questionnaire. It has been reported that the questionnaire takes 10 min to be completed [21]. SF-36 The SF-36 is a validated patient-administered, generic health-related quality of life (HRQL) instrument. It comprises 36 items across eight dimensions: physical functioning (PF), role-emotional functioning (RE), social functioning (SF), role physical functioning (RP), bodily pain (BP), vitality (VT), general health (GH), and mental health (MH) perceptions. Additional summary scores can be calculated from these domains for physical and mental health status (physical component score and mental
Knee Surg Sports Traumatol Arthrosc
component score). The eight dimensions of the SF-36 score are calculated on a 0–100, worst to best, scale [1, 28]. Visual analogue scale (VAS) Two 100-mm VASs were used to determine the severity of the ankle and foot pain (average pain over the last week) and disability (difficulty during daily activities over the past week). The VAS scores are presented on a 0–100, no pain or disability to most severe pain or disability, scale. American Orthopedic Foot and Ankle Society (AOFAS) Ankle and Hindfoot Scale The AOFAS Ankle Hindfoot Scale is one of the four rating systems developed by the AOFAS to provide a standard method of reporting the clinical status of the ankle and foot [14]. The AOFAS Ankle and Hindfoot Scale incorporates both patient-based and physical exam items, and converts the answer options into numerical scales to describe pain function and alignment. Scores are calculated on a 0–100, worst to best, scale. Forty points are assigned to pain, 50 to function, and 10 to alignment [14]. Statistical analysis Measurement properties such as reliability, construct validity, and interpretability were assessed as described below. Statistical analysis was performed by use of IBM SPSS 20.0 (SPSS Inc. Chicago, IL). Reliability Reliability of a questionnaire is the extent to which the measurement is free from measurement error [19]. Reliability in this study was assessed by internal consistency, test–retest reliability, and measurement error. Internal consistency is the degree of interrelatedness among the items of (sub) scales of the questionnaire [19]. Cronbach’s alpha was calculated to evaluate the internal consistency of the Dutch FAOS [4]. A Cronbach’s alpha between 0.7 and 0.95 was considered as satisfactory [25]. Additional unidimensionality of the subscales was checked using confirmatory factor analyses (CFA), as described in the section ‘Construct validity’. Test–retest reliability is the ability to measure the same way twice in patients with unchanged complains [19]. An additional question was added to the second questionnaire to assess whether the patient’s situation was stable. Patients who reported change in complaints were excluded from the test–retest analysis. Test–retest reliability was assessed by calculation of the intra class coefficients (ICCagreement,
two-way random-effects model) [8]. An ICC [ 0.7 was considered as good [24, 29]. Measurement error is the systematic and random error of a patient’s score that is not attributed to true changes in the construct(s) to be measured [19]. The standard error of measurement (SEM) was calculated as the square root of the within-subject variance (i.e. sum of the between-measures variance and the residual variance) [5]. Additionally, the smallest detectable change (SDC) was calculated from the SEM at individual level (1.96*H2*SEM) and at group level (SDC/Hn) [9]. Construct validity Validity is the degree to which the instrument measures the construct(s) it intends to measure [19]. In the absence of a gold standard, validity was assessed in terms of construct validity, which is the extent to which a score relates to other scores consistent with theoretically derived hypotheses [19]. Construct validity was evaluated by analysing the association between the FAOS subscales and the SF-36 domains, the AOFAS and the VASs by use of Spearman’s correlation coefficients. A coefficient between 0.4 and 0.7 (or -0.4 and -0.7) was defined as a moderate correlation, and coefficients lower than 0.4 (or higher than -0.4) were considered unrelated or measuring different constructs [3]. A priori hypotheses were formulated to evaluate construct validity; moderate correlations were expected between all five FAOS subscales with the physical SF-36 domain scores (PF, RP, BP), the physical component score (PCS), the AOFAS, and the VASs, and low correlations (\0.4) with the mental SF-36 domain scores (VT, RE, MH) and the mental component score (MCS). Confirmation of at least 75 % of the a priori hypotheses indicates sufficient construct validity [25]. Additional structural validity was assessed. This is the degree to which the scores of a HR-PRO instrument are an adequate reflection of the dimensionality of the construct to be measured [19]. For this purpose, CFA was performed using the FAOS data of all 250 participants. Model fit was evaluated by comparing the root mean square error of approximation (RMSEA), Comparative Fit Index (CFI), and Tucker–Lewis Fit Index (TLI). An RMSEA B0.06 indicated good model fit, and for CFI and TLI, a cut-off value of 0.95 was chosen [12, 23]. Interpretability Interpretability is the degree to which one can assign a qualitative meaning on a quantitative score or change in score [19]. One aspect of interpretability is the presence of floor and ceiling effects. If many patients score at the
123
Knee Surg Sports Traumatol Arthrosc
boundaries (worst and best possible score) of a questionnaire, the questionnaire can fail to demonstrate a decrease in patients demonstrating signs of clinical deterioration and vice versa. These floor and ceiling effects will limit its responsiveness and reliability. In this study, floor or ceiling effects were defined to be present if more than 15 % of the patients reported the worst or the best possible score [25].
missing from 83 questionnaires, which implies 1.4 % of all items, ranging from 0 (QOL) to 3.4 % (Sport) per domain. Calculation of sub scores could therefore not be performed for four domain scores (3 Sport and 1 Quality of Life) at baseline and two domain scores (all from the Sport domain) at retest. Reliability
Results One hundred and ten patients participated in the study. The population consisted of 59 (54 %) males and 51 (46 %) females with a mean age of 40 years (SD 13.6). Median duration of complaints was 36 (range 6–240) months, and 64 patients had undergone an operation, of which 20 cases were open and 44 involved arthroscopic surgery. Patients were diagnosed with hindfoot and ankle osteoarthritis (22 %), impingement (19 %), osteochondral defect or loose body (25 %), instability (4 %), tendinopathy (5 %), or other (25 %). A total of 83 (75 %) patients returned their second questionnaire, 8 patients reported a change in complaints and were not included in the test–retest reliability analysis. The FAOS scores of the first assessment are presented in Table 1. Missing data At baseline, 79 (1.7 %) from a total of 4,620 FAOS items (110 patients 9 42 items) were missing, ranging from 0.5 (Symptoms) to 4.0 % (Sport). At retest, 48 items were
Table 1 Descriptives and percentages highest and lowest scores of the FAOS, SF-36 domains, AOFAS, and VASs
PF physical function, RP role physical, BP bodily pain, GH general health, MCS mental component summary score, VT vitality, SF social function, RE role-emotional, and MH mental health, PCS physical component summary score, MCS mental component score
123
N = 110
Mean (SD)
FAOS pain
58.1 (19.9)
FAOS symptoms FAOS ADL
Cronbach’s alpha, intraclass correlation coefficients (ICC), SEM, and SDC are presented in Table 2. Unidimensionality of the domains was moderate, as described in the section ‘Construct validity’. The Cronbach’s alpha of the five subscales of the FAOS ranged from 0.76 (Symptoms) to 0.95 (ADL), which indicates a good internal consistency. For all subscales, the ICCs were between 0.83 (ADL) and Table 2 Reliability results of the FAOS FAOS pain
FAOS symptoms
FAOS ADL
FAOS sport/ Rec
FAOS QOL
Crohnbach’s alpha (n = 110)
0.91
0.76
0.95
0.87
0.77
ICC (n = 75)
0.85
0.88
0.83
0.87
0.86
SEM (n = 75)
6.5
6.6
7.7
7.7
6.5
SDC individual (n = 75)
18.0
18.4
21.4
21.5
18.2
2.1
2.1
2.5
2.5
2.1
SDC group (n = 75)
ICC intraclass correlation coefficient, SEM standard error of measurement, SDC smallest detectable change
Median (range)
Highest score (%)
Lowest score (%)
56 (6–100)
0
2 (1.8 %)
54.1 (21.1)
54 (7–96)
0
0
71.1 (19.8)
72 (17–100)
0
5 (4.5 %)
FAOS sport/Rec
37.4 (23.2)
35 (0–100)
1 (0.9 %)
2 (1.9 %)
FAOS QOL
30.0 (19.2)
31 (0–88)
7 (6.4 %)
0
SF-36 PF
57.5 (21.5)
55 (5–95)
0
0
SF-36 RP
45.9 (40.2)
50 (0–100)
34 (30.6 %)
31 (27.9 %)
SF-36 BP
46.5 (18.8)
41 (0–100)
3 (2.7 %)
1 (0.9 %)
SF-36 GH
69.6 (19.9)
72 (10–100)
0
7 (6.4 %)
SF-36 VT
61.1 (19.2)
60 (5–100)
0
3 (2.7 %)
SF-36 SF
73.5 (24.2)
75 (0–100)
2 (1.8 %)
31 (27.9 %)
SF-36 RE
77.8 (33.8)
100 (0–100)
11 (9.9 %)
68 (61.3 %)
SF-36 MH
75.3 (19.8)
76 (0–100)
1 (0.9 %)
17 (15.3 %)
VAS pain
51.5 (23.1)
55 (0–98)
1 (0.9)
0
VAS disability
53.8 (25.9)
60 (0–99)
2 (1.8 %)
0
AOFAS
57.0 (19.7)
60 (14–100)
0
1 (0.9 %)
Knee Surg Sports Traumatol Arthrosc Table 3 Spearman’s correlation coefficients (r) between FAOS scores and SF-36 subscale scores, AOFAS, and VASs (n = 110) FAOS pain
FAOS symptoms
FAOS ADL
FAOS sport/Rec
FAOS QOL
SF36-PF
0.62
0.47
0.67
0.63
0.54
SF36-RP
0.38
0.30
0.42
0.28
0.33
SF36-BP SF36-GH
0.60 0.18
0.36 0.07
0.57 0.21
0.57 0.09
0.60 0.22
SF36-VT
0.21
0.16
0.25
0.18
0.31
SF36-SF
0.39
0.37
0.49
0.40
0.49
SF36-RE
0.21
0.32
0.21
0.09
0.15
SF36-MH
0.35
0.24
0.35
0.24
0.38
SF36-PCS
0.55
0.32
0.59
0.54
0.53
SF36-MCS
0.22
0.27
0.25
0.15
0.26
AOFAS
0.56
0.33
0.44
0.47
0.47
AOFAS pain
0.55
0.33
0.44
0.41
0.39
AOFAS function
0.49
0.30
0.40
0.43
0.45
VAS pain
-0.66
-0.46
-0.53
-0.46
-0.63
VAS disability
-0.55
-0.38
-0.48
-0.48
-0.66
PF physical function, RP role physical, BP bodily pain, GH general health, MCS mental component summary score, VT vitality, SF social function, RE role-emotional, and MH mental health, PCS physical component summary score, MCS mental component score
0.88 (Symptoms). Test–retest reliability was therefore good. At the individual level, the SDC ranged from 18.0 (Pain) to 21.5 (Sport/Rec) and at group level from 2.1 (Pain, Symptoms, and Qol) to 2.5 (ADL and Sport/Rec) for the FAOS subscales (Table 2). Construct validity As hypothesized, a moderate correlation was observed between 4 FAOS subscales ‘Pain’, ‘ADL’, ‘Sport/Recreation’, and ‘QOL’ and the physical domains of the SF-36 (PF, BP), the PCS, the AOFAS, and both VASs (r [ 0.4 or r \ -0.4). However, correlations with the FAOS subscale ‘Symptoms’ and with the SF-36 domain Role Physical were generally lower than hypothesized (Table 3). The poor correlations between the 3 FAOS subscales for pain, ADL, and sport/recreation and the SF-36 domains concerning mental health (VT, RE, MH) and the MCS were confirmed (r \ 0.4). A total of 85 % of all a priori hypotheses was confirmed. CFA demonstrated a RMSEA of 0.09, a CFI of 0.92, and a TLI of 0.92, which indicated moderate model fit of the five-factor structure. Correlations among the five subscales ranged from 0.6 (Symptoms-ADL) to 0.8 (Pain-ADL).
Ceiling and floor effects For none of the individual FAOS-DLV domains, the percentage worst (0) or best (100) possible score had exceeded the predefined 15 %. Thus, no floor or ceiling effects were detected for any of the FAOS domains. However, clear floor and ceiling effects were observed for the SF-36 domains role physical, social function, role-emotional and mental health (Table 1).
Discussion The main results in this study revealed that both reliability and validity of the Dutch version of the FAOS were sufficient. The measurement properties of this Dutch language version of the FAOS were investigated in a population with varying ankle and hindfoot conditions. Good measurement properties of the questionnaire are warranted for precise measurement and evaluation of an intervention. To avoid confusing terminology concerning these measurement properties, the COSMIN checklist was used for reporting this study. As reported in other validation studies of the FAOS, only very few items were missing (1.6 % at baseline and 1.4 % at retest) [20, 21]. The subscale ‘Sport/Rec’ presented the most missing values (4 %). Nevertheless, scores of all subscales could be calculated. The internal consistency of all the subscales of the questionnaire was good. Cronbach’s alpha varied between 0.76 (Symptoms) and 0.95 (ADL) and was comparable to the results of other validation studies of the FAOS [20, 21]. The test–retest reliability (ICC) was high for all subscales of the FAOS, as was reported in all other validation studies of the FAOS [3, 13, 15, 20, 21]. Despite the good test–retest reliability, the SDC of all subscales was large and ranged between 18 and 21 points at individual level. This could be explained by the fact that the ICC is strongly influenced by the variability between the patients [9]. A heterogeneous population, as was included in our study, will yield a higher ICC than a homogeneous population. However, the SEMs found in our study (and therefore the SDC) were comparable to the SEMs reported in other validation studies of HR-PROs [6, 7]. The SEM is independent of the variability between the patients and is an important property for evaluative purposes in clinical settings [9]. The resulting SDC is the minimal change score that can be attributed to true change and not to measurement error. For group comparisons with our sample size of 75 patients, this value ranged between the 2.1 and 2.5 points. In case of monitoring a single patient overtime, an increase or decrease of at least 18–21.5 points has to be achieved to be interpreted as real change.
123
Knee Surg Sports Traumatol Arthrosc
Negabhan et al. [20] reported smaller SDCs in their validation study. However, they used another formula to calculate the SDC that could underestimate the value of the SDC. Terwee et al. [27] reported that it is likely that many PROMs are not suitable for evaluation at individual level due to large SEMs and are therefore not able to detect minimal but clinically important changes. However, at group level, the SDC is smaller, and with the right sample sizes, clinically important changes can be detected [27]. For proper reliability assessment, the patient should fill out the questionnaire twice under the same circumstances [18]. However, this criterion was not met in our study since the participants filled out the first questionnaire at the outpatient clinic and the second one at home. This could also attribute to the large SDC in our study. Despite some limited measurement properties, the SF36, AOFAS, and the VASs were the most appropriate instruments for the assessment of construct validity of the FAOS. They were available in Dutch language version, suitable for the study population, and enabled comparisons with other validation studies. Construct validity was sufficient considering the hypothesized moderate correlations between most of the subscales of the FAOS and the scales measuring similar concepts and poor correlations with those measuring dissimilar concepts. These results are comparable to the results of other validation studies [20, 21]. However, the absolute values of the correlation coefficients between the physical outcomes and the subscale ‘Symptoms’ were generally smaller than expected (\0.4). These results are comparable to those of Negahban et al. [20] and Chen et al. [3], who also reported low correlations between the SF36 domains and this subscale. Chen et al. [3] attributed these poor correlations to the foot-specific nature of the questions used in this subscale. In contrast, Karathepe et al. [13] presented correlations with the SF-36 domains BP, PF, and SF of 0.71, 0.73, and 0.53, respectively. It is possible that our hypothesis that this subscale was moderately correlated to other physical scales was too optimistic, since the subscale ‘Symptoms’ does comprise items such as swelling and crepitation, aspects that do not necessarily have to be related to physical function or pain. However, considering our small values of the correlation coefficients, the subscale ‘Symptoms’ was still stronger associated with physical outcome measures than with the domain scores concerning mental health. Additionally, model fit of the five-factor structure was only moderate, which could be explained by the high correlations between the subscales within the questionnaire itself. More research is needed to make proper conclusion on the unidimensionality of the constructs. Similar to other studies [13, 20], the percentage of patients having the worst or best possible score did not
123
exceed the predefined 15 %, demonstrating the absence of floor and ceiling effects for all subscales of the FAOS. However, Roos et al. [21] reported substantial ceiling effects for all subscales of the FAOS, all exceeding 15 %, which they attributed to the long period of 12 years after ligament reconstruction of the ankle. The main limitation of this study is that pre- and postinterventional assessments were not incorporated in the study design. As a consequence, neither the responsiveness nor the minimal important change in the FAOS-DLV could be assessed. Information about these measurement properties is required to be able to evaluate the appropriateness for evaluative trials and to calculate sample sizes, and should be assessed in additional studies. The clinical relevance of this validation study is that it confirms that the Dutch version of the FAOS is valid and comparable to the available versions in other languages, making the questionnaire appropriate for clinical trials and other evaluation purposes. The obtained results demonstrate that the questionnaire can be used for patients with a variety of ankle and hindfoot conditions.
Conclusion The Dutch language version of the FAOS (FAOS-DLV) seems to have adequate measurement properties for use in patients with varying ankle and hindfoot conditions. The questionnaire is, however, more suitable for clinical evaluation at group level than for monitoring a specific patient. Further research can clarify the unidimensionality of the subscales of the FAOS-DLV. Acknowledgments The authors would like to thank Marlijn de Beer for her outstanding work in patient recruitment and data management, and John D. King for reading and editing the manuscript. Conflict of interest of interest.
The authors declare that they have no conflict
References 1. Aaronson NK, Muller M, Cohen PD, Essink-Bot ML, Fekkes M, Sanderman R, Sprangers MA, Te Velde A, Verrips E (1998) Translation, validation, and norming of the Dutch language version of the SF-36 Health Survey in community and chronic disease populations. J Clin Epidemiol 51:1055–1068 2. Button G, Pinney S (2004) A meta-analysis of outcome rating scales in foot and ankle surgery: is there a valid, reliable, and responsive system? Foot Ankle Int 25:521–525 3. Chen L, Lyman S, Do H, Karlsson J, Adam SP, Young E, Deland JT, Ellis SJ (2012) Validation of Foot and Ankle Outcome Score for hallux valgus. Foot Ankle Int 33:1145–1155 4. Cronbach LJ (1951) Coefficient alpha and the internal structure of tests. Psychometrika 16:297–334
Knee Surg Sports Traumatol Arthrosc 5. De Boer MR, de Vet HCW, Terwee CB, Moll AC, Vo¨lker-Dieben HJM, van Rens GHMB (2005) Changes to the subscales of two vision-related quality of life questionnaires are proposed. J Clin Epidemiol 58:1260–1268 6. De Groot IB, Favejee MM, Reijman M, Verhaar JA, Terwee CB (2008) The Dutch version of the knee injury and osteoarthritis outcome score: a validation study. Health Qual Life Outcomes 6:16 7. De Groot IB, Reijman M, Terwee CB, Bierma-Zeinstra SMA, Favejee M, Roos EM, Verhaar JAN (2007) Validation of the Dutch version of the Hip disability and Osteoarthritis Outcome Score. Osteoarthr Cartil 15:104–109 8. De Vet HCW, Terwee CB, Knol DL, Bouter LM (2006) When to use agreement versus reliability measures. J Clin Epidemiol 59:1033–1039 9. De Vet HC, Terwee CB, Ostelo RW, Beckerman H, Knol DL, Bouter LM (2006) Minimal changes in health status questionnaires: distinction between minimally detectable change and minimally important change. Health Qual Life Outcomes 4:54 10. Eechaute C, Vaes P, Van Aerschot L, Asman S, Duquet W (2007) The clinimetric qualities of patient-assessed instruments for measuring chronic ankle instability: a systematic review. BMC Musculoskelet Disord 8:6 11. Guillemin F, Bombardier C, Beaton D (1993) Cross-cultural adaptation of health-related quality of life measures: literature review and proposed guidelines. J Clin Epidemiol 46:1417–1432 12. Hu L, Bentler P (1999) Cutoff criteria for fit indexes in covariance structure analysis: conventional criteria versus new alternatives. Struct Equ Model 6:1–55 ¨ zbek G (2009) 13. Karatepe AG, Gu¨naydin R, Kaya T, Karlibas¸ U, O Validation of the Turkish version of the foot and ankle outcome score. Rheumatol Int 30:169–173 14. Kitaoka HB, Alexander IJ, Adelaar RS, Nunley JA, Myerson MS, Sanders M (1994) Clinical rating systems for the ankle-hindfoot, midfoot, hallux, and lesser toes. Foot Ankle Int 15:349–353 15. Mani SB, Brown HC, Nair P, Chen L, Do HT, Lyman S, Deland JT, Ellis SJ (2013) Validation of the Foot and Ankle Outcome Score in Adult Acquired Flatfoot Deformity. Foot Ankle Int 34:1140–1146 16. Martin RL, Irrgang JJ (2007) A survey of self-reported outcome instruments for the foot and ankle. J Orthop Sports Phys Ther 37:72–84 17. Martin RL, Irrgang JJ, Lalonde KA, Conti S (2006) Current concepts review: foot and ankle outcome instruments. Foot Ankle Int 27:383–390
18. Mokkink LB, Terwee CB, Patrick DL, Alonso J, Stratford PW, Knol DL, Bouter LM, De Vet HCW (2010) The COSMIN checklist for assessing the methodological quality of studies on measurement properties of health status measurement instruments: an international Delphi study. Qual Life Res 19:539–549 19. Mokkink LB, Terwee CB, Patrick DL, Alonso J, Stratford PW, Knol DL, Bouter LM, de Vet HCW (2010) The COSMIN study reached international consensus on taxonomy, terminology, and definitions of measurement properties for health-related patientreported outcomes. J Clin Epidemiol 63:737–745 20. Negahban H, Mazaheri M, Salavati M, Sohani SM, Askari M, Fanian H, Parnianpour M (2010) Reliability and validity of the foot and ankle outcome score: a validation study from Iran. Clin Rheumatol 29:479–486 21. Roos EM, Brandsson S, Karlsson J (2001) Validation of the foot and ankle outcome score for ankle ligament reconstruction. Foot Ankle Int 22:788–794 22. Roos EM, Roos HP, Ekdahl C, Lohmander LS (1998) Knee injury and Osteoarthritis Outcome Score (KOOS)—validation of a Swedish version. Scand J Med Sci Sports 8:439–448 23. Schreiber JB, Nora A, Stage FK, Barlow EA, King J (2006) Reporting structural equation modeling and confirmatory factor analysis results: a Review. J Educ Res 99:323–337 24. Streiner DL, Norman GR (2008) Health measurement scales: a practical guide to their development and use. In: 2nd (ed) Oxford University Press, New York 25. Terwee CB, Bot SDM, de Boer MR, van der Windt DAWM, Knol DL, Dekker J, Bouter LM, de Vet HCW (2007) Quality criteria were proposed for measurement properties of health status questionnaires. J Clin Epidemiol 60:34–42 26. Terwee CB, Mokkink LB, Knol DL, Ostelo RWJG, Bouter LM, de Vet HCW (2012) Rating the methodological quality in systematic reviews of studies on measurement properties: a scoring system for the COSMIN checklist. Qual Life Res 21:651–657 27. Terwee CB, Roorda LD, Knol DL, De Boer MR, De Vet HCW (2009) Linking measurement error to minimal important change of patient-reported outcomes. J Clin Epidemiol 62:1062–1067 28. Ware JE, Sherbourne CD (1992) The MOS 36-item short-form health survey (SF-36). I. Conceptual framework and item selection. Med Care 30:473–483 29. Weir JP (2005) Quantifying test-retest reliability using the intraclass correlation coefficient and the SEM. J Strength Cond Res 19:231–240
123
