The Journal of Arthroplasty xxx (2015) xxx–xxx

Contents lists available at ScienceDirect

The Journal of Arthroplasty journal homepage: www.arthroplastyjournal.org

Construct Validity and Test Re-Test Reliability of the Forgotten Joint Score Simon M. Thompson, MBBS, MD(Res), FRCS(Tr & Orth), Lucy J. Salmon, BappSci(Physio), PhD , Justin M. Webb, MBBS, FRACS, Leo A. Pinczewski, MBBS, FRACS, Justin P. Roe, MBBS, FRACS North Sydney Orthopaedic and Sports Medicine Centre, The Mater Clinic, Suite 2, Wollstonecraft NSW, Australia

a r t i c l e

i n f o

Article history: Received 17 December 2014 Accepted 5 May 2015 Available online xxxx Keywords: Forgotten Joint Score arthroplasty assessment validity reliability

a b s t r a c t Consecutive patients undergoing knee arthroplasty completed questionnaires: FJS, Knee Injury and Osteoarthritis Outcome Score (KOOS) and WOMAC Score (mean 39 months after surgery), and were mailed a repeat questionnaire after 4 to 6 weeks. The test–retest reliability was almost perfect for the FJS (ICC = 0.97), and the FJS subdomains (ICC > 0.8). Convergent construct validity of the FJS was correlated with the KOOS Subscores of Quality of Life (0.63, P = 0.001), Symptom (0.33, P = 0.001), Pain (0.68, P = 0.001) and ADL (0.66, P = 0.001) and the Total WOMAC (0.70, P = 0.001). The FJS demonstrates high test–retest reliability and construct validity compared to the Normalised WOMAC and KOOS Subscales. The FJS does not demonstrate the ceiling effect of the WOMAC or KOOS pain scores so may have greater discriminatory ability following TKR. © 2015 Elsevier Inc. All rights reserved.

Total knee arthroplasty is an ever increasingly utilised surgical treatment of osteoarthritis [1]. Whilst the overall success of TKA as a treatment modality for osteoarthritis is not in question, there remain a significant number of patients who remain dissatisfied with their arthroplasty [2,3]. Outcomes following total knee arthroplasty are typically assessed using clinical scoring tools. Many different scoring tools are available including both patient and clinician based tools, disease specific (WOMAC) and global health questionnaires (SF-12). These scores should be validated, reproducible and responsive to changes in the patient’s condition [4]. Despite many of these tools having been validated and tested extensively there is no commonly accepted ‘gold standard’ measurement tool to assess TKA outcomes [5]. As patient outcomes have continued to improve and patient expectations have increased over recent years, traditionally used scoring tools have begun to demonstrate a ceiling effect, potentially losing the ability to determine differences in outcome in a high functional range [6,7]. Additionally, from a patient’s perspective the true success of the procedure may not equate to the sum of a set outcome variables. It has recently been suggested that the ultimate goal of arthroplasty surgery is for the patient to be able to forget their prosthetic joint during regular day to

One or more of the authors of this paper have disclosed potential or pertinent conflicts of interest, which may include receipt of payment, either direct or indirect, institutional support, or association with an entity in the biomedical field which may be perceived to have potential conflict of interest with this work. For full disclosure statements refer to http://dx.doi.org/10.1016/j.arth.2015.05.001. Reprint requests: Lucy J Salmon, BappSci(Physio), PhD, North Sydney Orthopaedic and Sports Medicine Centre, The Mater Clinic, Suite 2, 3 Gillies St Wollstonecraft NSW 2065 Australia.

day activities. A new scoring system, The Forgotten Joint Score (FJS) has been developed. The FJS focuses on the patients’ awareness of their joint arthroplasty during a range of day to day and recreational activities [8]. The FJS has been validated in German and French language form, but not in English [9]. This score consists of 12 questions where subjects are asked to rate their awareness of their joint arthroplasty during various activities. The aim of this study was to investigate the test retest reliability and the construct validity of the FJS-12 in English, specifically for patients who have undergone total knee arthroplasty. Materials and Methods Patients Ethical approval for this study was sought and granted from an independent ethical review board. All patients provided signed informed consent to participate. From a prospective surgical knee registry, 240 patients who had undergone primary total knee arthroplasty under the care of a single surgeon between 2006 and 2010 were invited via mail to participate in the study. Patients with a significant cognitive impairment, an intellectual disability or mental illness were excluded. Patients were asked to complete a questionnaire consisting of the Forgotten Joint Score (FJS-12) and the Knee Injury and Osteoarthritis Outcome Score (KOOS). A normal knee (normal range of motion, no pain or instability on a day to day basis) should be the benchmark test for a forgotten joint arthroplasty. The FJS-12 evaluates the knee according to the patients’ ability to forget their joint arthroplasty in comparison to a normal knee in everyday life, by assessing variables such as functional ability, expectations, activity level, psychosocial factors, stiffness and pain. The Western Ontario and McMaster Universities

http://dx.doi.org/10.1016/j.arth.2015.05.001 0883-5403/© 2015 Elsevier Inc. All rights reserved.

Please cite this article as: Thompson SM, et al, Construct Validity and Test Re-Test Reliability of the Forgotten Joint Score, J Arthroplasty (2015), http://dx.doi.org/10.1016/j.arth.2015.05.001

2

S.M. Thompson et al. / The Journal of Arthroplasty xxx (2015) xxx–xxx

Table 1 Statistics for the FJS, WOMAC and KOOS Scales.

FJS-12 Normalised WOMAC

KOOS

Initial Follow up Pain Stiffness Function Total Quality of Life Symptom Pain ADL

Mean

SD

Range

62 60 90 84 90 90 76 65 90 89

29 29 13 17 11 11 18 14 12 13

0–100 0–100 50–100 38–100 53–100 52–100 12–100 4–100 56–100 35–100

Table 2 Test Retest Reliability Analysis of the FJS-12.

Overall Score Individual Questions Night Symptoms Sitting Walking Bathing Travelling Stairs Walking uneven ground Rising Standing Housework/Gardening Walking/Hiking Favourite Sport

Intra Class Correlation

95% Confidence Interval

Landis & Koch Classification

0.97

0.95–0.98

Almost Perfect

0.88 0.84 0.92 0.88 0.86 0.94 0.91 0.90 0.91 0.91 0.92 0.94

0.84–0.92 0.77–0.88 0.89–0.94 0.84–0.92 0.81–0.90 0.92–0.96 0.88–0.94 0.86–0.93 0.88–0.94 0.87–0.93 0.89–0.94 0.91–0.96

Almost Perfect Almost Perfect Almost Perfect Almost Perfect Almost Perfect Almost Perfect Almost Perfect Almost Perfect Almost Perfect Almost Perfect Almost Perfect Almost Perfect

of each subscale and dividing by the maximum total score for the scale. Those who completed and returned their initial questionnaire were mailed a repeat questionnaire at 4 weeks. Statistical Analysis Mean and standard deviations were calculated for each of the measurement variables. Test retest reliability was calculated using the interclass correlation coefficient (ICC). This was calculated for the overall score and for the individual components and was classified according to Landis and Koch’s guidelines of almost perfect (N0.8), substantial (0.6–0.8), moderate (0.4–0.6), fair (0.2–0-4), slight (0.0–0.2) and poor (b 0.0). Convergent construct validity was assessed with a Spearman’s correlation between the first FJS score and the normalised WOMAC scales and the KOOS scales. Statistical significance was set at 0.05. Statistical analysis was performed with SPSS Version 11. Results A total of 147 of the 240 (61%) completed and returned both questionnaires and were included in the analysis. There were 68 females and 79 males with a mean age of 67 years (range 32–89). The right knee was involved in 75 cases. A Triathalon (Stryker) prosthesis was used in 120 knees and a Genesis II (Smith & Nephew) prosthesis was used in 27 knees. The mean time from surgery to completion of the first questionnaire was 39 months (range 18–72). The FJS-12 returned a mean score of 62 and 60 (range 0–100) and the normalised WOMAC overall mean score of 90 (range 52–100). Results are summarised in Table 1. Test–Retest Reliability of the FJS-12 The test–retest reliability for the Forgotten Joint Score is shown in Table 2. The mean time between completion of questionnaires was 6 weeks (range 3–15).

Table 3 Correlation of the FJS-12 with the Normalised WOMAC and KOOS Scales. Forgotten Joint Score

Normalised WOMAC

KOOS

Pain Stiffness Function Total Quality of Life Symptom Pain ADL

Spearman’s

Significance

0.67 0.52 0.66 0.70 0.63 0.33 0.68 0.66

0.001 0.001 0.001 0.001 0.001 0.007 0.001 0.001

(WOMAC) osteoarthritis index scores were calculated from the KOOS score, and normalised in to a scale where high scores indicate a good outcome. WOMAC scores were normalised by summing the total score

Convergent Construct Validity of FJS Convergent construct validity of the FJS-12 was compared to both the KOOS and WOMAC subscales using correlation (Spearman’s correlation coefficient). The results are shown in Table 3. The distribution of the FJS to the Normalised WOMAC and KOOS Subscales is shown in the Figs. 1–5. It can be seen in Figs. 1–5 that the Forgotten Joint Score demonstrates a much greater distribution of responses than the WOMAC or KOOS pain scores, which are clustered at the ceiling. The percentage of subjects who reported the maximum score is a measure of the ceiling effect. The FJS-12 showed a ceiling effect of 6.8% (10/147) and a floor of 3.4% (5/147). The ceiling effect was 2% for KOOS symptom, 13% for KOOS Quality of Life, 33% for KOOS Pain score and 9% for the normalised WOMAC score.

Fig. 1. Distribution of the Forgotten Joint Score.

Please cite this article as: Thompson SM, et al, Construct Validity and Test Re-Test Reliability of the Forgotten Joint Score, J Arthroplasty (2015), http://dx.doi.org/10.1016/j.arth.2015.05.001

S.M. Thompson et al. / The Journal of Arthroplasty xxx (2015) xxx–xxx

Fig. 2. Distribution of Normalised WOMAC Scores.

Fig. 3. Distribution of KOOS Symptom Scores.

Fig. 4. Distribution of KOOS Quality of Life Scores.

Fig. 5. Distribution of KOOS Pain Scores.

Please cite this article as: Thompson SM, et al, Construct Validity and Test Re-Test Reliability of the Forgotten Joint Score, J Arthroplasty (2015), http://dx.doi.org/10.1016/j.arth.2015.05.001

3

4

S.M. Thompson et al. / The Journal of Arthroplasty xxx (2015) xxx–xxx

Discussion No universally accepted definition for the ultimate success of total knee arthroplasty currently exists. It can be argued that the ability of a patient to ‘forget’ that they have a joint prosthesis is close to the ultimate goal for joint arthroplasty surgery. The FJS-12 is a newly developed outcome tool that evaluates a patient’s ability to forget their joint prosthesis during a variety of everyday activities such as walking, bathing and in bed. The FJS was shown to have almost perfect test retest reliability with the overall intra-class correlation of 0.97. Each individual question also showed almost perfect test retest reliability with the lowest intra-class correlation of 0.84 for awareness when sitting for more than an hour. This is similar to reported reliability testing of the KOOS [10]. The FJS-12 showed good convergent construct validity as shown by high correlations with the WOMAC scores. Our result of r = 0.7 is similar to that reported by Behrend in the initial validation study (0.79) [8]. We also tested the construct validity against the KOOS. Good correlation was also shown for the quality of life, pain and activity of daily living (ADL) scores but not for the symptoms score. One possible explanation for this is the functional focus of the questions of the FJS-12. Of the 12 questions that comprise the score only 2 relate to awareness of the knee at rest. The quality of life, pain and ADL score questions of the KOOS also have functional emphasis whereas the questions of the symptoms score relate more to general symptoms rather than during particular activities. As this study was conducted retrospectively there were no preoperative data to enable an assessment of the responsiveness of the FJS-12 that is, its ability to measure changes in patients’ function and symptoms. However other authors have found that the FJS-12 was more responsive than Knee Society Score Knee score and WOMAC after total knee arthroplasty [11]. When a measure possesses a distinct upper limit and when a large concentration of participants scorse at or near this limit, this is known as a ceiling effect. The opposite is the floor effect. The FJS-12 showed an absolute ceiling effect of 6.8% (10/147) and a floor of 3.4% (5/147), in-

dicating that a perfect score is obtained rarely. It can be seen from Figs. 1–5 that the FJS-12 demonstrated a wider distribution of responses than the other scores, and so may be more sensitive to changes over time and better able to discriminate differences between groups. The FJS demonstrates high test retest reliability and construct validity compared to the normalised WOMAC and KOOS Subscales. The FJS does not demonstrate the ceiling effect of the WOMAC or KOOS pain scores and so may have greater discriminatory ability following total knee arthroplasty. Acknowledgements Research Support for this study was received from Stryker, Inc. References 1. Australian Orthopaedic Association National Joint Replacement R. Annual report. Australian Orthopaedic Association, National Joint Replacement Registry; 2011. 2. Hawker G, Wright J, Coyte P, et al. Health-related quality of life after knee replacement. Results of the Knee Replacement Patient Outcomes Research Team Study*. J Bone Joint Surg 1998;80(2):163. 3. Noble PC, Conditt MA, Cook KF, et al. The John Insall Award: patient expectations affect satisfaction with total knee arthroplasty. Clin Orthop Relat Res 2006;452:35. 4. Irrgang JJ, Anderson AF, Boland AL, et al. Development and validation of the International Knee Documentation Committee Subjective Knee Form. Am J Sports Med 2001; 29(5):600. 5. Bourne RB. Measuring tools for functional outcomes in total knee arthroplasty. Clin Orthop Relat Res 2008;466(11):2634. 6. Marx RG, Jones EC, Atwan NC, et al. Measuring improvement following total hip and knee arthroplasty using patient-based measures of outcome. J Bone Joint Surg Am 2005;87(9):1999. 7. Na S-E, Ha C-W, Lee C-H. A new high-flexion knee scoring system to eliminate the ceiling effect. Clin Orthop Relat Res 2012;470(2):584. 8. Behrend H, Giesinger K, Giesinger JM, et al. The "forgotten joint" as the ultimate goal in joint arthroplasty: validation of a new patient-reported outcome measure. J Arthroplasty 2012;27(3):430. 9. Thienpont E, Opsomer G, Koninckx A, et al. Joint awareness in different types of knee arthroplasty evaluated with the Forgotten Joint Score. J Arthroplasty 2014;29(1):48. 10. Roos EM, Toksvig-Larsen S. Knee injury and Osteoarthritis Outcome Score (KOOS) — validation and comparison to the WOMAC in total knee replacement. Health Qual Life Outcomes 2003;1:17. 11. Giesingeremail K, Hamiltonemail DF, Jostemail B, et al. Comparative responsiveness of outcome measures for total knee arthroplasty. Osteoarthr Cartil 2014;22(2):184.

Please cite this article as: Thompson SM, et al, Construct Validity and Test Re-Test Reliability of the Forgotten Joint Score, J Arthroplasty (2015), http://dx.doi.org/10.1016/j.arth.2015.05.001

Construct Validity and Test Re-Test Reliability of the Forgotten Joint Score.

Consecutive patients undergoing knee arthroplasty completed questionnaires: FJS, Knee Injury and Osteoarthritis Outcome Score (KOOS) and WOMAC Score (...
584KB Sizes 2 Downloads 21 Views