http://informahealthcare.com/dre ISSN 0963-8288 print/ISSN 1464-5165 online Disabil Rehabil, Early Online: 1–10 ! 2015 Informa UK Ltd. DOI: 10.3109/09638288.2015.1024340
RESEARCH PAPER
Rehabilitation Complexity Scale: Italian translation and transcultural validation Lisa Galletti1, Maria Grazia Benedetti2, Serena Maselli2, Gustavo Zanoli3, Elettra Pignotti4, and Roberto Iovine1
Disabil Rehabil Downloaded from informahealthcare.com by Nyu Medical Center on 06/08/15 For personal use only.
1
Physical Medicine and Rehabilitation Unit, San Giovanni in Persiceto, Bologna, Italy, 2Physical Medicine and Rehabilitation Unit, Istituto Ortopedico Rizzoli, Bologna, Italy, 3GLOBE & Cochrane MSG, University of Ferrara, Ferrara, Italy, and 4Department of Statistical Science ‘‘Paolo Fortunati’’, University of Bologna, Bologna, Italy Abstract
Keywords
Purpose: The aim of the present study was translation, cultural adaption and validation of the extended version 12 of the Rehabilitation Complexity Scale (RCS-E) in a sample of patients with stroke and total hip replacement. Method: The cross-cultural validation required RCS-E forward– backward translation, revision by an expert committee and its application in an Intensive Rehabilitation setting through a retrospective collection of data from clinical records. The evaluation of the psychometric properties was carried out by analyzing the correlations between RCS-E score and other measures (Functional Independence Measure, Braden, Morse, Cumulative Illness Rating Scale) and the assessment of reliability in terms of reproducibility (inter-observer agreement) and repeatability (intra-observer agreement). Results: The backward and forward processes of translation of the scale did not create problems of interpretation of terms. Some adaptation was required for the items nursing (N), medical care (M) and therapeutic intensity (TI) due to differences on the national health system structure. The Italian version of the scale proved to be valid, reliable with high reproducibility and repeatability. Conclusions: The Italian version RCS-E has been successfully validated, showing good psychometric properties, which partly reproduce the results obtained for the original version. However, some assumption was made for some items thus preventing possible comparison with other countries.
Italian validation, needs assessment, psychometric properties, Rehabilitation Complexity Scale History Received 1 September 2014 Revised 30 January 2015 Accepted 25 February 2015 Published online 15 April 2015
ä Implications for Rehabilitation
Admittance at an Intensive Rehabilitation care setting in Italy requires to evaluate the complexity of rehabilitation needs. The Rehabilitation Complexity Scale (RCS-E) has proved to be reliable for assessing clinical complexity and consequently for planning rehabilitation needs. The Italian version of RCS-E has been successfully validated, showing good psychometric properties, which reproduce the results obtained for the original version. The items included in the therapy intensity subscale do not fit the Italian health system rules for intensity of rehabilitation care and needs adaptation.
Introduction The appropriateness in healthcare processes is a very timely topic, with particular emphasis on rehabilitation settings individuation [1]. Appropriateness is defined by WHO as a complex issue with various dimensions and definitions and these differ between countries. However, most definitions of appropriateness address a number of key requirements: that care is effective (based on valid evidence); efficient (cost-effectiveness); and consistent with the
Address for correspondence: Maria Grazia Benedetti, Physical Therapy and Rehabilitation Unit, Istituto Ortopedico Rizzoli, Via Pupilli 1, 40136 Bologna, Italy. Tel: +39 0516366236. Fax: +39 051332392. E-mail: [email protected]
ethical principles and preferences of the relevant individual, community or society. In order to not make appropriateness appear more value laden and qualitative than efficacy, effectiveness, efficiency (cost-effectiveness) and empowerment, economists have attempted to introduce quantitative definitions of inappropriateness. From this perspective, ‘‘underuse’’ occurs when a cost-effective intervention with a proven net benefit is not performed, ‘‘overuse’’ occurs when an intervention which is of unproven net benefit or which is not cost effective is performed and ‘‘misuse’’ occurs when an intervention which has a negative net benefit is performed [2]. According to a recent publication [3], the cost of inappropriateness is 21–34% of the global healthcare cost; the cost of inappropriateness in excess would vary between 6% and 8.5% of global healthcare spending, while the cost of care services not provided or performance of low quality (inappropriateness in
Disabil Rehabil Downloaded from informahealthcare.com by Nyu Medical Center on 06/08/15 For personal use only.
2
L. Galletti et al.
default) would be between 5% and 7.5% of total healthcare cost; administrative complexity, uncontrolled prices and fraud would affect the remaining part. In order to correctly define the degree of needs that patients undergoing rehabilitation might have and assign them to the most appropriate rehabilitation setting, the recent Italian Guidelines for Rehabilitation [4] identify three factors: disability, comorbidity and complexity. Whereas consolidated and commonly used tools are available for the assessment of the first two factors, respectively, the Functional Independence Measure (FIM) scale [5] or the Barthel Index [6] for disability, and the Cumulative Illness Rating Scale (CIRS) [7] for comorbidity, the latter factor, i.e. complexity, is still the most difficult to define and assess. The few tools available to measure complexity are not widespread and universally accepted. Assessing the complexity of rehabilitation needs for individuating the appropriate care setting is, in fact, a worldwide challenge [8–10]. In the USA, Canada, Australia and most of Europe, the rehabilitation complexity classification is based on the concept of physical dependence (measured by the FIM scale or Barthel Index) [11,12] as a surrogate measure of rehabilitation complexity. These classifications may work reasonably well where physical independence is the main target of intervention. However, they do not capture needs for medical or specialist nursing care, nor do they specifically address the need for cognitive, behavioural or other psychological interventions. In 2007, the Rehabilitation Complexity Scale (RCS) was introduced to fill this gap [13] and proved to be reliable for assessing clinical complexity and consequently for planning rehabilitation needs and appropriateness of rehabilitation settings [14]. The latest published version of the RCS is the version 12 extended [15], which includes five levels of clinical-rehabilitation complexity (with the corresponding cost refund systems), which are C ¼ need for basic care and support (0–4); N ¼ nursing needs (0–3); T ¼ nature and intensity of each intervention therapy (TD 0–4, TI 0–4 ! 0–8); M ¼ need for medical intervention (0–3) and E ¼ need for equipment (0–2) with a total score that can range from 0 to 20. The extended version 12 of the Rehabilitation Complexity Scale (RCS-E) is now adopted as the principal case mix measure within the service commissioning dataset for specialist rehabilitation services in England, where serial rating of the RCS-E scores form the basis of a multi-level weighted per diem payment model where reimbursement changes over time as the patient improves [16]. The aim of the present study was the Italian translation, transcultural adaptation, retrospective applicability and validation (validity, reliability) of RCS-E in the rehabilitation context of the authors’ country.
Materials and methods The RCS was used initially by its authors [13] to measure the differences between two types of rehabilitation services (tertiary ‘‘complex specialized’’ (CS) services intended for patients with highly complex needs and ‘‘district specialist’’ services intended for patients with less complex needs) in terms of patient complexity and rehabilitation service supplied, with the aim of offering a more suitable tool to determine the refund fees in the rehabilitation field. That study showed clear differences, in terms of RCS score, between the two types of rehabilitation services, with a median cross-sectional score 9 as indicative of more specialized services and more intense care at that time. Successively, the authors [14] assessed the ‘‘clinimetric’’ properties of the Rehabilitation Complexity Scale version 2
Disabil Rehabil, Early Online: 1–10
(RCS-v2) in a sample of patients admitted to neuro-rehabilitation in a context of CS services. The latest published version of the RCS was version 12 – extended (RCS-E) [15] which differs from the previous version (RCS-v2) by extending the range of the two therapy subscales (TD and TI) to 0–4; the inclusion of a Risk item (R) as an alternative to Care (C) (also on a scale of 0–4) and the insertion of a new item addressing needs for Equipment (E 0–2). The authors demonstrated that RCS-E addresses the ceiling effects of the RCS and provides more information compared with the previous version in identifying patients with highly complex rehabilitation needs. Recently a new version, the RCS-E (13th version) has been proposed by the authors which also extends the range of the Medical and Nursing Scales to 0–4. Although not yet published nor validated in the original language, an Italian translation and a preliminary reliability study has been carried out [17]. Translation and trans-cultural validation The trans-cultural validation of the RCS (RCS-E 12) consisted of the following phases: Phase 1 – Forward translation: Translation from English to Italian of the RCS-E 12 and the further instructions for application attached to RCS-v2. The aim was to retain the concepts of the original scale, using culturally and clinically fitting expressions. Two translations were performed independently by translators with Italian as their native tongue, with wide experience in the scope of rehabilitation and with a fair knowledge of the scale. A colloquial language was maintained and basically the translation of the terms was literal compared with original version. The uncertain terms were discussed and resolved between the two translators. Phase 1 ended when a common adaption was shared. None of the items of the scale was excluded. Concerning the translation of the further instructions for application attached to version 2 of the scale, the same principles described for translating the scale itself were followed. Only items of the most recent version (RCS-E) which were not present in the previous version (RCS-v2) were added, with the aim of obtaining scale and instructions that matched in number and properties of the items. Phase 2 – Backward translation: Two bilingual native Englishspeaking translators, without knowledge of the scale, backward retranslated the initial translation. Taking into account cultural and language differences, the conceptual equivalence and vocabulary differences, the aim was to make sure that the Italian version reflected the same items contents of the original version. Phase 3 – Expert committee: The translated versions were submitted to a bilingual committee, composed of the four translators (R.I., M.G.B., B.M., K.S.) and a methodologist (G.Z.). The committee assessed the equivalence of the items between the original version and the Italian version from a conceptual and semantic point of view, with the aim of identifying difficulties, inconsistencies and mistakes in translation. Phase 3 ended when a definitive Italian version was achieved. Phase 4 – Test of the tool in the target setting: The final version of the scale translated into Italian (RCS-E-It) was applied retrospectively to a pilot study of 100 clinical files, to test the psychometric properties for the validation. Phase 5 – Submission to the developer: The final version of the Italian translation was sent to the author of the original scale. Study design The assessment of the Italian version of the RCS (RCS-E-It) was based on a retrospective pilot study, which involved calculations of the RCS-E score on 100 clinical files: 50 of post-acute patients
Validation of RCS-E Italian version
DOI: 10.3109/09638288.2015.1024340
discharged from the intensive care unit of Rehabilitation Medicine of San Giovanni in Persiceto Hospital with a primary diagnosis of stroke and 50 patients undergoing total hip replacement and discharged from the intensive care unit of Rehabilitation Medicine of Istituto Ortopedico Rizzoli of Bologna. The clinical files consisted of the last 100 (50 for each care unit) consecutive files from the date of authorization of the study by the local Ethical Committee. This choice was motivated by the intention to capture the true complexity of a sample of patients previously hospitalized, and the presence of the necessary information in the medical records.
Disabil Rehabil Downloaded from informahealthcare.com by Nyu Medical Center on 06/08/15 For personal use only.
Psychometric properties of the scale Psychometric properties of the scale were explored according to the methods used for the validation of the original scale [14]. The validity of the scale was assessed by analyzing correlations between the RCS-E-It score obtained on admittance into the ward (both for each partial item and the total score) and other indicators of appropriateness used in Italy (FIM motor, cognitive and total score on admittance and at discharge), by using Spearman’s rank correlation coefficient r (rho). Furthermore, in order to check the ability of RCS-E-It to capture the case complexity, it was correlated by using Spearman’s rank correlation coefficient r (rho) to other scores usually registered in our rehabilitation wards upon patient admission: the Braden score (to evaluate the risk of compromising skin integrity) [18], the Morse score (used to evaluate the risk of accidental falls) [19] and the CIRS [7] (for co-morbidities assessment). To assess the reproducibility of the scale, the files were blindly viewed simultaneously and independently by two researchers (G.L., S.M.). To ensure the necessary uniformity in the assessment of the files, the first five (for each care unit) were assessed together by the two researchers to establish the methodology of filling in the data collection form. These five files were excluded from the overall assessment of the study, thus allowing inter-rater agreement to be calculated. After about 3 months, the two researchers re-assessed separately and independently the same files to assess test–retest reliability. From the information contained in the clinical files, the two researchers allotted the partial and total scores of the RCS-E-It. Agreement in allotting the score of the partial RCS-E-It items was assessed by contingency tables and Cohen’s kappa coefficient. Conversely, agreement of the RCS-E-It total score was assessed by calculating the Interclass Correlation Coefficient (ICC) based on one way random model with measures of absolute agreement. All statistical analyses were performed with SPSS (SPSS Inc., Chicago, IL) and p50.05 was the accepted level of significance.
Results Transcultural translation and validation The agreed Italian version of RCS-E-It is attached in Appendix 1. No particular problems were encountered in the forward translation. Only few differences in linguistic (stated/indicated, investigation/testing, professional nurse/qualified nurse, etc.) were found in the backward translation. They were discussed and considered to not affect the meaning of the statement. Rather, the main problem was relative to the application of the RCS-E-It to the Italian health system, which is largely different from the English one and required some assumptions. Major differences in the item ‘‘nursing’’ were identified because nursing education in Italy is not based on the acquisition of a specialization in a particular field of medical practice
3
(e.g. rehabilitation, mental health, palliative care, neuro-nurse, etc.) but this specialization comes only from practice in a particular department or area, or more rarely by specific postgraduate certificates. Therefore, since in Italian rehabilitation units there is actually only one type of nurse, the only difference that we could put in evidence in data collection was that of ‘‘general registered nursing’’ and ‘‘rehab-trained nurses’’. We considered the former to be less specialized and therefore dedicated to those patients who do not require special postural attentions, gait training or particular help regarding difficulties in communication (e.g. aphasia and dysarthria) but that need assistance by a general nurse. Similar problems also arose with the item ‘‘medical care’’; and also in this case it was necessary to adapt the item to the Italian healthcare system. In particular, the Italian hospital system is based on specialized wards with continued specialized medical care supplied by consultants and not by junior physicians. So, in order to adapt the item M to our system, we have considered the usual stay in a rehabilitation ward as M2, while M1 refers to areas of less intensive medical/rehabilitation care and M3 to potentially unstable conditions/emergency situations. This is very different from the UK health system where the M score provides more granularity in assessing the individual medical needs of the patients at the time, even while they remain in a rehabilitation setting. Thus, the RCS-M as applied in the UK provides a patientlevel estimation of individual patient needs as these change over time, while the Italian system rates at service level (Intensive Rehabilitation versus Extensive Rehabilitation). Finally, regarding the item ‘‘intensity of rehabilitation treatment (TI)’’, since the delivery system of rehabilitation treatment in Italy is less variable than the English one, the following adaptation was applied: TI1 was considered as an ‘‘extensive’’ rehabilitation treatment in Italy (at least 1 h/day, as delivered for example in less intensive settings), TI2 was assimilated to an ‘‘intensive’’ rehabilitation treatment (at least 3 h/day, as delivered in a rehabilitation ward), TI3 was considered as TI2 but with a therapist plus an assistant and TI4 with two qualified therapists. Descriptive statistics The overall mean age of the patients was 67.83 years (range 40–90). Thirty-eight percent of the patients were male and 62% female. The mean CIRS was 3.79 (range 0–8), Braden score was 18.34 (range 11–22) and Morse score was 32.66 (range 0–95). Descriptive statistics for the RCS-E-It and FIM scores at admission and discharge are reported in Table 1. The RCS-E-It scores at admission had a median score of 10 (range 8–14), FIM motor 51 (range 13–88) and FIM cognitive 35 (range 5–35). At discharge the RCS-E-It scores median score was 6 (range 4–12), FIM motor was 77 (range 13–90) and FIM cognitive was 35 (range 5–35). The items RCS-C, RCS-N and RCS-M were decreased at discharge. Particularly, it is worth to remark that the decreasing of the item RCS-M at discharge was due to the progressive medical improvement of the patients so that they did no more require a high intensive specialized setting, but they could be discharged to a less intensive setting or at home. The item RCS-TD increased due the needs to include even more different therapy disciplines along the hospital stay with respect to the admission. RCS-TI and RCS-E items remained quite stable based on the single intensive care setting explored, and the same need of aids along the hospital stay. Validity A weak correlation between the RCS-E-It total score and subscales at admission and FIM score (motor and cognitive), CIRS, Braden and Morse scores was found (Table 2).
4
L. Galletti et al.
Disabil Rehabil, Early Online: 1–10
Table 1. Descriptive statistics for scores at admission and discharge. Admission (n ¼ 100)
RCS C (0–4) RCS N (0–3) RCS M (0–3) RCS TD (0–4) RCS TI (0–4) RCS E (0–2) RCS TOT (0–20) Motor FIM Cognitive FIM Total FIM
Discharge (n ¼ 100)
Wilcoxon signed rank test
Mean ± SD
Median
Range
Mean ± SD
Median
Range
z
p
1.47 ± 0.86 1.96 ± 0.24 2.02 ± 0.14 1.29 ± 0.46 3.04 ± 0.37 0.98 ± 0.14 10.75 ± 1.24 43.04 ± 19.25 28.46 ± 9.58 71.50 ± 26.84
1 2 2 1 3 1 10 51 35 86
0–3 1–3 2–3 1–2 2–4 0–1 8–14 13–88 5–35 18–115
0.66 ± 0.97 0.23 ± 0.42 0.08 ± 0.27 1.72 ± 0.78 2.90 ± 0.50 0.91 ± 0.32 6.50 ± 2.09 64.25 ± 24.24 30.01 ± 8.20 94.26 ± 31.01
0 0 0 2 3 1 6 77 35 112
0–3 0–1 0–1 1–3 0–4 0–2 4–12 13–90 5–35 18–122
8.16 9.15 9.62 5.85 2.27 2.33 8.73 8.42 4.63 8.41
50.0005 50.0005 50.0005 50.0005 0.23 0.02 50.0005 50.0005 50.0005 50.0005
Disabil Rehabil Downloaded from informahealthcare.com by Nyu Medical Center on 06/08/15 For personal use only.
Table 2. Spearman correlations between RCS scores and other indicators at admission.
RCS N RCS M RCS TD RCS TI RCS E RCS TOT FIM motor FIM cognitive CIRS Braden Morse
RCS C
RCS N
RCS M
RCS TD
RCS TI
RCS E
RCS TOT
FIM motor
FIM cognitive
CIRS
Braden
0.24* 0.17 0.37*** 0.12 0.18 0.82*** 0.79*** 0.72*** 0.55*** 0.75*** 0.22*
0.02 0.02 0.02 0.27** 0.40*** 0.34** 0.08 0.05 0.17 0.07
0.16 0.01 0.01 0.18 0.16 0.19 0.03 0.11 0.04
0.13 0.22* 0.59*** 0.33** 0.55*** 0.22* 0.51*** 0.21*
0.02 0.26** 0.01 0.14 0.01 0.15 0.09
0.18 0.24* 0.13 0.04 0.03 0.06
0.75*** 0.65*** 0.46*** 0.70*** 0.25*
0.70*** 0.62*** 0.76*** 0.26*
0.64*** 0.73*** 0.29**
0.65*** 0.12
0.34**
*p50.05, **p50.01, ***p50.001.
A negative correlation was found between the RCS-E-It total score and subscales C, N, TD and E and the FIM motor score. RCS-E-It total score and subscales C and TD were inversely correlated also to cognitive FIM. Total RCS-E-It score and subscales C and TD were correlated with number of co-morbidities (according to the CIRS scale) and also with the Morse score and the Braden score. Reliability Inter-rater agreement/reproducibility: The ICC, concerning the total RCS-E-It score, was 0.882 (p50.0005). With respect to the partial items of the scale, the classification identity rates between the two observers, Cohen’s kappa coefficient values and the p significance are shown in Table 3. While a substantial agreement was found for subscales C, M, TD and S, the agreement for TI was moderate and for N even fair. Looking at the identification rate, an agreement was, however, present, respectively, in 96% and 86% of cases. Test–retest repeatability: the ICC concerning the total RCS-EIt score was 0.93 (p50.0005) for observer 1 and 0.77 (p50.0005) for observer 2. Classification identity rates between the two observers for each observer, Cohen’s kappa coefficient values and the p significance are shown in Table 3. Agreement for subscales ranged from substantial to almost perfect for both the observers. However, observer 2 tended to attribute the same score in the two assessment sessions for most of the subscales. The lack of Cohen’s kappa coefficient values and significance for items C, N, M and E is thus attributable to the total absence of variability of the sample and the consequent impossibility to perform statistical analyses.
Table 3. Inter-rater and test–retest agreement.
ICC Cronbach alpha p RCS C K cohen Id p RCS N K cohen Id p RCS M K cohen Id p RCS TD K cohen Id p RCS TI K cohen Id p RCS E K cohen Id p
Inter-rater agreement
Test–retest agreement obs1
Test–retest agreement obs2
0.882 0.944 50.0005
0.93 0.968 50.0005
0.772 0.876 50.0005
0.69 86% 5 0,0005
0.779 90% 50.0005
1 100% 50.0005
0.267 95% 0.001
0.793 98% 50.0005
– 100% –
0.662 99% 50.0005
0.999 100% 0.01
– 100% –
0.749 85% 50.0005
0.814 89% 50.0005
0.672 92% 50.0005
0.582 96% 50.0005
0.732 0.960 50.0005
0.847 98% 50.0005
0.152 91% 0.041
0.651 96% 50.0005
– 100% –
DOI: 10.3109/09638288.2015.1024340
Disabil Rehabil Downloaded from informahealthcare.com by Nyu Medical Center on 06/08/15 For personal use only.
Discussion The present study was aimed at providing Italian translation, transcultural validation and analysis of the psychometric properties (retrospective applicability, validity and reliability) of the validated RCS-E on a sample of patients with stroke sequelae and total hip arthroplasty admitted to an Intensive Rehabilitation Ward. According to guidelines’ recommendations [20,21], the transcultural validation was successfully performed through the translation of the scale in Italian, its back translation, the revision by a committee of experts and its testing in the clinical setting to ensure that the meaning of the original version was kept also in the translation. The translation of the RCS-E 12th version presented some problems in comparing the UK rehabilitation system and the Italian system, thus requiring an adaptation of the translation. Main differences were relative to (1) the level of specialization of nurses, which does not formally exist in Italy, (2) the medical needs, which in Italy are essentially based on different services with respect to the intensity of rehabilitation required and hence (3) the intensity level of rehabilitation care that is well stated in most of the Italian regions, and determines the reimbursement from the National Health System. Intensive Rehabilitation wards provide a minimum of 3 h/day, at least 5 days a week to complex patients with higher rehabilitation needs, while Extensive Rehabilitation wards provide a rehabilitative time ranging from 1 to 3 h/day to patients who are less demanding from a rehabilitation point of view but still require hospital care. This distinction actually is not applicable to the Intensity of Treatment in the original scale: TI1 which corresponds to a ‘‘Low level – less than daily (e.g. assessment/review/maintenance/supervision) OR Group therapy only’’ does not correspond exactly in fact to the Extensive Rehabilitation in Italy, as well as TI2 that is ‘‘Moderate – daily intervention – individual sessions with one person to treat for most sessions OR very intensive Group program of 6 h/day’’ does not correspond exactly to the Intensive Rehabilitation in our country. Furthermore TI3 ‘‘High level – daily intervention with therapist PLUS assistant and/or additional group sessions’’ and TI4 ‘‘Very High level – very intensive (e.g. two trained therapists to treat, or total 1:1 therapy 430 h/week)’’ are not represented in the usual rehabilitation care in Italian National Health System. Based on these considerations, it must be taken into account that the assumptions made and the application of the RCS-E in the Italian context could inevitably lead to ceiling effects which are a product of the scoring rules, rather than the instrument itself thus impacting on comparability between countries. The RCS-E-It scale proved to be retrospectively applicable to all the clinical files examined. Although the first publication of the original RCS score in 2007 suggested that a median of 9/15 as a cut-off for complex specialised services [14], this was a cross-sectional analysis including patients at all stages of their rehabilitation programme, including those nearing discharge. Recent figures from the UK Rehabilitation Outcomes Collaborative (based on data collected 2010–2014) record a mean of 12.2 (SD 2.6) for RCS-E-v12 on admission to Level 1 Complex Specialised Services, and of 11.3 (SD 2.5) for admission to a Level 2 service (Personal communication Professor Turner-Stokes). Therefore, the mean value of RCS-E found in the present study probably corresponds to a lower intensity rehabilitation service in UK. This can be explained by the presence in the sample of patients without significant cognitive/communication deficits (the Median FIM Cognitive score is 35, i.e. the maximum score). Moreover, half of the patients were undergoing fairly simple orthopedic rehabilitation
Validation of RCS-E Italian version
5
after a hip replacement (which would be managed in a Level 3 service in the UK). Results obtained in the orthopedic sample could suggest the need for a more appropriate setting characterized by a lower rehabilitation complexity for these patients. In agreement with the findings of clinimetric properties of RCS 2 [14] and RCS-E [15], the comparison with the FIM score provided weak correlations with RCS-E-It, especially concerning the RCS-E scores related to the need for medical care (item M) and therapeutic needs (items TD + TI). As stated by TurnerStokes [15] comparison of RCS-E-It with the FIM score shows how this disability scale provides good indications on the need for basic and nursing care, but is a poor indicator of therapeutic and medical care needs. Particularly, the expected negative correlation between the RCS-E-It score and the FIM score as well as the statistically significant correlation obtained with CIRS, Braden score and Morse score confirms the validity of the scale in assessing complexity of patients from basic care, nursing, medical and therapy needs. However, as none of these are direct measures of therapy or medical needs, only a weak correlation between the RCS-E-It and these scales was found. Unfortunately, it was not possible to exactly reproduce the validation of the original scale because Northwick Park Nursing and Therapy Dependency Scales are not used in our country [14]. Anyway, within the available comparisons, we believe that RCS-E-It behaved similarly to the original, confirming its value in assessing the complexity of rehabilitation needs and its complementary role among other outcome measures. The ICC showed for the total RCS-E-It score an excellent inter-rater agreement between the two observers (0.882). This is particularly relevant because, as stated by Bland and Altman [22], while values of 0.7–0.8 are regarded as satisfactory for scales used as research tools, a minimum of 0.90 is desirable for clinical applications. A lower agreement rate was found for subscales, which ranged from fair agreement for N to substantial agreement for other subscales mostly confirming findings of original RCS-E version 2 validations [15]. Actually, looking at data, only in 5% of cases the agreement on N score was not present, confirming, however, some difficulty in identifying correctly the type (N1/N2/N3) of nursing assistance. In the test–retest agreement, the ICC was excellent for observer 1 (0.93), while a lower ICC (0.77) was found for observer 2. In this case, statistical results for subscales were affected by the absence of variation in many answers of this observer for most of the subscales. For both observers, all subscales showed an agreement ranging from substantial to almost perfect. A limitation of the present study is the small number of patients and the case mix, which differed from the validation study of the original scale, including also orthopedic patients. Considering that the clinimetric properties of RCS-E were already previously assessed by other authors [14,15], the small sample has been judged adequate for the transcultural validation. Another potential weakness of our study design was the retrospective application of the scale, which is not the way it is supposed to be used in clinical setting. This choice, however, granted us some advantages for the validation purposes, namely the fact of having complete and stable information from the clinical records for all cases, allowing us to perform retest for reliability after a considerable amount of time, thus minimising memory effect. However, it also implies some disadvantages, given that not all interventions may have been fully recorded. This could have introduced a bias, under-rating the activities and reducing the opportunity for comparison with samples where the RCS is scored in real time by clinicians. Furthermore, this additional step between the patient and the rater may affect inter-rater reliability,
6
L. Galletti et al.
and scoring discrepancies may reflect observer differences in interpretation of the clinical records, rather than differences in the use of the rating scale itself. In conclusion, the Italian validation of the RCS-E confirmed that the scale has good psychometric properties, in particular in terms of validity and repeatability. However, a particular adaptation has been required for subscales ‘‘Nursing’’, ‘‘Medical’’ and ‘‘Therapy Intensity’’ for cultural Italian health system structure differences with the UK system which could prevent comparison. Further studies, with particular attention to the adapted subscales, to homogeneous groups of pathologies, and with a prospective application of the method, will be needed to make RCS-E-It a routinely used tool to assess the appropriateness of the rehabilitation setting in Italy.
Disabil Rehabil Downloaded from informahealthcare.com by Nyu Medical Center on 06/08/15 For personal use only.
Acknowledgements The authors are grateful to Keith Smith and Brett Martyn for the back translation of the RCS-E-It scale. Professor Lynne Turner Stokes, developer of the RCS scale, is acknowledged for her invaluable help with the critical interpretation of the study results.
Declaration of interest The authors report no declaration of interest.
References 1. Castaldi S, Bevilacqua L, Arcari G, et al. How appropriate is the use of rehabilitation facilities? Assessment by an evaluation tool based on the AEP protocol. J Prev Med Hyg 2010;51:116–20. 2. World Health Organization. Appropriateness in health care services. Report on a WHO Workshop; 2000 Mar 23–25; Koblenz, Germany. 3. Berwick DM, Hackbarth AD. Eliminating waste in US health care. JAMA 2012;307:1513–16. 4. Ministero della Salute. Piano d’indirizzo per la riabilitazione – Suppl. Ord. N. 60, G.U. n . 50 del 2/3/2011. Available from: http:// www.salute.gov.it [last accessed 31 Aug 2014]. 5. Heinemann AW, Linacre JM, Wright BD, et al. Relationships between impairment and physical disability as measured by the Functional Independence Measure. Arch Phys Med Rehabil 1993; 74:566–73. 6. Mahoney FI, Barthel DW. Functional evaluation: the Barthel Index. Md State Med J 1965;14:61–5.
Disabil Rehabil, Early Online: 1–10
7. Linn BS, Linn MV, Gurel L. Cumulative illness rating scale. J Am Geriatr Soc 1968;16:622–6. 8. Guile´ R, Leux C, Paille´ C, et al. Validation of a tool assessing appropriateness of hospital days in rehabilitation centres. Int J Qual Health Care 2009;21:198–205. 9. Poulos CJ, Eagar K. Determining appropriateness for rehabilitation or other subacute care: is there a role for utilisation review? Aust New Zealand Health Policy 2007;4:1–7. 10. Poulos CJ, Magee C, Bashford G, Eagar K. Determining level of appropriateness in the patient journey from acute care to rehabilitation. BMC Health Service Res 2011;11:1–9. 11. Sutherland JM, Walker J. Challenges of rehabilitation case mix measurement in Ontario hospitals. Health Policy 2008;85:336–48. 12. Eagar K. The Australian National Sub-Acute and Non-Acute Patient casemix classification. Aust Health Rev 1999;22:180–96. 13. Turner-Stokes L, Disler R, Williams H. The Rehabilitation Complexity Scale: a simple, practical tool to identify ‘complex specialized’ services in neurological rehabilitation. Clin Med 2007; 7:593–9. 14. Turner-Stokes L, Williams H, Siegert RJ. The Rehabilitation Complexity Scale version 2: a clinimetric evaluation in patient with severe complex neurodisability. J Neurol Neurosurg Psychiatry 2010;81:146–53. 15. Turner-Stokes L, Scott H, Williams H, Siegert R. The Rehabilitation Complexity Scale – extended version: detection of patient with highly complex needs. Disabil Rehabil 2012;34:715–20. 16. Turner-Stokes L, Sutch S, Dredge R. Healthcare tariffs for specialist inpatient neurorehabilitation services: rationale and development of a UK casemix and costing methodology. Clin Rehab 2012;26: 264–79. 17. Roda F, Agosti M, Corradini E, et al. Cross-cultural adaptation and preliminary test–retest reliability of the Italian version of the Complexity Rehabilitation Scale-Extended (13th Version). Eur J Phys Rehabil Med 2014 Mar 04. [Epub ahead of print]. 18. Bergstrom N, Braden BJ, Laguzza A, Holman V. The Braden Scale for predicting pressure sore risk. Nurs Res 1987;36:205–10. 19. Morse JM, Morse RM, Tylko SJ. Development of a scale to identify the fall-prone patient. Can J Aging 1989;8:366–7. 20. Beaton D, Bombardier C, Guilleman F, et al. Recommendations for the cross-cultural adaptation of health status measures. Rosemont (IL): American Academy of Orthopaedic Surgeons, Institute for Work and Health; 1998. 21. Beaton DE, Bombardier C, Guillemin F, et al. Guidelines for the process of cross-cultural adaptation of self-report measures. Spine 2000;25:3186–91. 22. Bland JM, Altman DG. Statistics notes: Cronbach’s alpha. BMJ 1997;314:572.
Disabil Rehabil Downloaded from informahealthcare.com by Nyu Medical Center on 06/08/15 For personal use only.
DOI: 10.3109/09638288.2015.1024340
Appendix 1.
Validation of RCS-E Italian version 7
Disabil Rehabil Downloaded from informahealthcare.com by Nyu Medical Center on 06/08/15 For personal use only.
8 L. Galletti et al. Disabil Rehabil, Early Online: 1–10
Disabil Rehabil Downloaded from informahealthcare.com by Nyu Medical Center on 06/08/15 For personal use only.
DOI: 10.3109/09638288.2015.1024340
Validation of RCS-E Italian version 9
Disabil Rehabil Downloaded from informahealthcare.com by Nyu Medical Center on 06/08/15 For personal use only.
10 L. Galletti et al. Disabil Rehabil, Early Online: 1–10
RESEARCH PAPER
Rehabilitation Complexity Scale: Italian translation and transcultural validation Lisa Galletti1, Maria Grazia Benedetti2, Serena Maselli2, Gustavo Zanoli3, Elettra Pignotti4, and Roberto Iovine1
Disabil Rehabil Downloaded from informahealthcare.com by Nyu Medical Center on 06/08/15 For personal use only.
1
Physical Medicine and Rehabilitation Unit, San Giovanni in Persiceto, Bologna, Italy, 2Physical Medicine and Rehabilitation Unit, Istituto Ortopedico Rizzoli, Bologna, Italy, 3GLOBE & Cochrane MSG, University of Ferrara, Ferrara, Italy, and 4Department of Statistical Science ‘‘Paolo Fortunati’’, University of Bologna, Bologna, Italy Abstract
Keywords
Purpose: The aim of the present study was translation, cultural adaption and validation of the extended version 12 of the Rehabilitation Complexity Scale (RCS-E) in a sample of patients with stroke and total hip replacement. Method: The cross-cultural validation required RCS-E forward– backward translation, revision by an expert committee and its application in an Intensive Rehabilitation setting through a retrospective collection of data from clinical records. The evaluation of the psychometric properties was carried out by analyzing the correlations between RCS-E score and other measures (Functional Independence Measure, Braden, Morse, Cumulative Illness Rating Scale) and the assessment of reliability in terms of reproducibility (inter-observer agreement) and repeatability (intra-observer agreement). Results: The backward and forward processes of translation of the scale did not create problems of interpretation of terms. Some adaptation was required for the items nursing (N), medical care (M) and therapeutic intensity (TI) due to differences on the national health system structure. The Italian version of the scale proved to be valid, reliable with high reproducibility and repeatability. Conclusions: The Italian version RCS-E has been successfully validated, showing good psychometric properties, which partly reproduce the results obtained for the original version. However, some assumption was made for some items thus preventing possible comparison with other countries.
Italian validation, needs assessment, psychometric properties, Rehabilitation Complexity Scale History Received 1 September 2014 Revised 30 January 2015 Accepted 25 February 2015 Published online 15 April 2015
ä Implications for Rehabilitation
Admittance at an Intensive Rehabilitation care setting in Italy requires to evaluate the complexity of rehabilitation needs. The Rehabilitation Complexity Scale (RCS-E) has proved to be reliable for assessing clinical complexity and consequently for planning rehabilitation needs. The Italian version of RCS-E has been successfully validated, showing good psychometric properties, which reproduce the results obtained for the original version. The items included in the therapy intensity subscale do not fit the Italian health system rules for intensity of rehabilitation care and needs adaptation.
Introduction The appropriateness in healthcare processes is a very timely topic, with particular emphasis on rehabilitation settings individuation [1]. Appropriateness is defined by WHO as a complex issue with various dimensions and definitions and these differ between countries. However, most definitions of appropriateness address a number of key requirements: that care is effective (based on valid evidence); efficient (cost-effectiveness); and consistent with the
Address for correspondence: Maria Grazia Benedetti, Physical Therapy and Rehabilitation Unit, Istituto Ortopedico Rizzoli, Via Pupilli 1, 40136 Bologna, Italy. Tel: +39 0516366236. Fax: +39 051332392. E-mail: [email protected]
ethical principles and preferences of the relevant individual, community or society. In order to not make appropriateness appear more value laden and qualitative than efficacy, effectiveness, efficiency (cost-effectiveness) and empowerment, economists have attempted to introduce quantitative definitions of inappropriateness. From this perspective, ‘‘underuse’’ occurs when a cost-effective intervention with a proven net benefit is not performed, ‘‘overuse’’ occurs when an intervention which is of unproven net benefit or which is not cost effective is performed and ‘‘misuse’’ occurs when an intervention which has a negative net benefit is performed [2]. According to a recent publication [3], the cost of inappropriateness is 21–34% of the global healthcare cost; the cost of inappropriateness in excess would vary between 6% and 8.5% of global healthcare spending, while the cost of care services not provided or performance of low quality (inappropriateness in
Disabil Rehabil Downloaded from informahealthcare.com by Nyu Medical Center on 06/08/15 For personal use only.
2
L. Galletti et al.
default) would be between 5% and 7.5% of total healthcare cost; administrative complexity, uncontrolled prices and fraud would affect the remaining part. In order to correctly define the degree of needs that patients undergoing rehabilitation might have and assign them to the most appropriate rehabilitation setting, the recent Italian Guidelines for Rehabilitation [4] identify three factors: disability, comorbidity and complexity. Whereas consolidated and commonly used tools are available for the assessment of the first two factors, respectively, the Functional Independence Measure (FIM) scale [5] or the Barthel Index [6] for disability, and the Cumulative Illness Rating Scale (CIRS) [7] for comorbidity, the latter factor, i.e. complexity, is still the most difficult to define and assess. The few tools available to measure complexity are not widespread and universally accepted. Assessing the complexity of rehabilitation needs for individuating the appropriate care setting is, in fact, a worldwide challenge [8–10]. In the USA, Canada, Australia and most of Europe, the rehabilitation complexity classification is based on the concept of physical dependence (measured by the FIM scale or Barthel Index) [11,12] as a surrogate measure of rehabilitation complexity. These classifications may work reasonably well where physical independence is the main target of intervention. However, they do not capture needs for medical or specialist nursing care, nor do they specifically address the need for cognitive, behavioural or other psychological interventions. In 2007, the Rehabilitation Complexity Scale (RCS) was introduced to fill this gap [13] and proved to be reliable for assessing clinical complexity and consequently for planning rehabilitation needs and appropriateness of rehabilitation settings [14]. The latest published version of the RCS is the version 12 extended [15], which includes five levels of clinical-rehabilitation complexity (with the corresponding cost refund systems), which are C ¼ need for basic care and support (0–4); N ¼ nursing needs (0–3); T ¼ nature and intensity of each intervention therapy (TD 0–4, TI 0–4 ! 0–8); M ¼ need for medical intervention (0–3) and E ¼ need for equipment (0–2) with a total score that can range from 0 to 20. The extended version 12 of the Rehabilitation Complexity Scale (RCS-E) is now adopted as the principal case mix measure within the service commissioning dataset for specialist rehabilitation services in England, where serial rating of the RCS-E scores form the basis of a multi-level weighted per diem payment model where reimbursement changes over time as the patient improves [16]. The aim of the present study was the Italian translation, transcultural adaptation, retrospective applicability and validation (validity, reliability) of RCS-E in the rehabilitation context of the authors’ country.
Materials and methods The RCS was used initially by its authors [13] to measure the differences between two types of rehabilitation services (tertiary ‘‘complex specialized’’ (CS) services intended for patients with highly complex needs and ‘‘district specialist’’ services intended for patients with less complex needs) in terms of patient complexity and rehabilitation service supplied, with the aim of offering a more suitable tool to determine the refund fees in the rehabilitation field. That study showed clear differences, in terms of RCS score, between the two types of rehabilitation services, with a median cross-sectional score 9 as indicative of more specialized services and more intense care at that time. Successively, the authors [14] assessed the ‘‘clinimetric’’ properties of the Rehabilitation Complexity Scale version 2
Disabil Rehabil, Early Online: 1–10
(RCS-v2) in a sample of patients admitted to neuro-rehabilitation in a context of CS services. The latest published version of the RCS was version 12 – extended (RCS-E) [15] which differs from the previous version (RCS-v2) by extending the range of the two therapy subscales (TD and TI) to 0–4; the inclusion of a Risk item (R) as an alternative to Care (C) (also on a scale of 0–4) and the insertion of a new item addressing needs for Equipment (E 0–2). The authors demonstrated that RCS-E addresses the ceiling effects of the RCS and provides more information compared with the previous version in identifying patients with highly complex rehabilitation needs. Recently a new version, the RCS-E (13th version) has been proposed by the authors which also extends the range of the Medical and Nursing Scales to 0–4. Although not yet published nor validated in the original language, an Italian translation and a preliminary reliability study has been carried out [17]. Translation and trans-cultural validation The trans-cultural validation of the RCS (RCS-E 12) consisted of the following phases: Phase 1 – Forward translation: Translation from English to Italian of the RCS-E 12 and the further instructions for application attached to RCS-v2. The aim was to retain the concepts of the original scale, using culturally and clinically fitting expressions. Two translations were performed independently by translators with Italian as their native tongue, with wide experience in the scope of rehabilitation and with a fair knowledge of the scale. A colloquial language was maintained and basically the translation of the terms was literal compared with original version. The uncertain terms were discussed and resolved between the two translators. Phase 1 ended when a common adaption was shared. None of the items of the scale was excluded. Concerning the translation of the further instructions for application attached to version 2 of the scale, the same principles described for translating the scale itself were followed. Only items of the most recent version (RCS-E) which were not present in the previous version (RCS-v2) were added, with the aim of obtaining scale and instructions that matched in number and properties of the items. Phase 2 – Backward translation: Two bilingual native Englishspeaking translators, without knowledge of the scale, backward retranslated the initial translation. Taking into account cultural and language differences, the conceptual equivalence and vocabulary differences, the aim was to make sure that the Italian version reflected the same items contents of the original version. Phase 3 – Expert committee: The translated versions were submitted to a bilingual committee, composed of the four translators (R.I., M.G.B., B.M., K.S.) and a methodologist (G.Z.). The committee assessed the equivalence of the items between the original version and the Italian version from a conceptual and semantic point of view, with the aim of identifying difficulties, inconsistencies and mistakes in translation. Phase 3 ended when a definitive Italian version was achieved. Phase 4 – Test of the tool in the target setting: The final version of the scale translated into Italian (RCS-E-It) was applied retrospectively to a pilot study of 100 clinical files, to test the psychometric properties for the validation. Phase 5 – Submission to the developer: The final version of the Italian translation was sent to the author of the original scale. Study design The assessment of the Italian version of the RCS (RCS-E-It) was based on a retrospective pilot study, which involved calculations of the RCS-E score on 100 clinical files: 50 of post-acute patients
Validation of RCS-E Italian version
DOI: 10.3109/09638288.2015.1024340
discharged from the intensive care unit of Rehabilitation Medicine of San Giovanni in Persiceto Hospital with a primary diagnosis of stroke and 50 patients undergoing total hip replacement and discharged from the intensive care unit of Rehabilitation Medicine of Istituto Ortopedico Rizzoli of Bologna. The clinical files consisted of the last 100 (50 for each care unit) consecutive files from the date of authorization of the study by the local Ethical Committee. This choice was motivated by the intention to capture the true complexity of a sample of patients previously hospitalized, and the presence of the necessary information in the medical records.
Disabil Rehabil Downloaded from informahealthcare.com by Nyu Medical Center on 06/08/15 For personal use only.
Psychometric properties of the scale Psychometric properties of the scale were explored according to the methods used for the validation of the original scale [14]. The validity of the scale was assessed by analyzing correlations between the RCS-E-It score obtained on admittance into the ward (both for each partial item and the total score) and other indicators of appropriateness used in Italy (FIM motor, cognitive and total score on admittance and at discharge), by using Spearman’s rank correlation coefficient r (rho). Furthermore, in order to check the ability of RCS-E-It to capture the case complexity, it was correlated by using Spearman’s rank correlation coefficient r (rho) to other scores usually registered in our rehabilitation wards upon patient admission: the Braden score (to evaluate the risk of compromising skin integrity) [18], the Morse score (used to evaluate the risk of accidental falls) [19] and the CIRS [7] (for co-morbidities assessment). To assess the reproducibility of the scale, the files were blindly viewed simultaneously and independently by two researchers (G.L., S.M.). To ensure the necessary uniformity in the assessment of the files, the first five (for each care unit) were assessed together by the two researchers to establish the methodology of filling in the data collection form. These five files were excluded from the overall assessment of the study, thus allowing inter-rater agreement to be calculated. After about 3 months, the two researchers re-assessed separately and independently the same files to assess test–retest reliability. From the information contained in the clinical files, the two researchers allotted the partial and total scores of the RCS-E-It. Agreement in allotting the score of the partial RCS-E-It items was assessed by contingency tables and Cohen’s kappa coefficient. Conversely, agreement of the RCS-E-It total score was assessed by calculating the Interclass Correlation Coefficient (ICC) based on one way random model with measures of absolute agreement. All statistical analyses were performed with SPSS (SPSS Inc., Chicago, IL) and p50.05 was the accepted level of significance.
Results Transcultural translation and validation The agreed Italian version of RCS-E-It is attached in Appendix 1. No particular problems were encountered in the forward translation. Only few differences in linguistic (stated/indicated, investigation/testing, professional nurse/qualified nurse, etc.) were found in the backward translation. They were discussed and considered to not affect the meaning of the statement. Rather, the main problem was relative to the application of the RCS-E-It to the Italian health system, which is largely different from the English one and required some assumptions. Major differences in the item ‘‘nursing’’ were identified because nursing education in Italy is not based on the acquisition of a specialization in a particular field of medical practice
3
(e.g. rehabilitation, mental health, palliative care, neuro-nurse, etc.) but this specialization comes only from practice in a particular department or area, or more rarely by specific postgraduate certificates. Therefore, since in Italian rehabilitation units there is actually only one type of nurse, the only difference that we could put in evidence in data collection was that of ‘‘general registered nursing’’ and ‘‘rehab-trained nurses’’. We considered the former to be less specialized and therefore dedicated to those patients who do not require special postural attentions, gait training or particular help regarding difficulties in communication (e.g. aphasia and dysarthria) but that need assistance by a general nurse. Similar problems also arose with the item ‘‘medical care’’; and also in this case it was necessary to adapt the item to the Italian healthcare system. In particular, the Italian hospital system is based on specialized wards with continued specialized medical care supplied by consultants and not by junior physicians. So, in order to adapt the item M to our system, we have considered the usual stay in a rehabilitation ward as M2, while M1 refers to areas of less intensive medical/rehabilitation care and M3 to potentially unstable conditions/emergency situations. This is very different from the UK health system where the M score provides more granularity in assessing the individual medical needs of the patients at the time, even while they remain in a rehabilitation setting. Thus, the RCS-M as applied in the UK provides a patientlevel estimation of individual patient needs as these change over time, while the Italian system rates at service level (Intensive Rehabilitation versus Extensive Rehabilitation). Finally, regarding the item ‘‘intensity of rehabilitation treatment (TI)’’, since the delivery system of rehabilitation treatment in Italy is less variable than the English one, the following adaptation was applied: TI1 was considered as an ‘‘extensive’’ rehabilitation treatment in Italy (at least 1 h/day, as delivered for example in less intensive settings), TI2 was assimilated to an ‘‘intensive’’ rehabilitation treatment (at least 3 h/day, as delivered in a rehabilitation ward), TI3 was considered as TI2 but with a therapist plus an assistant and TI4 with two qualified therapists. Descriptive statistics The overall mean age of the patients was 67.83 years (range 40–90). Thirty-eight percent of the patients were male and 62% female. The mean CIRS was 3.79 (range 0–8), Braden score was 18.34 (range 11–22) and Morse score was 32.66 (range 0–95). Descriptive statistics for the RCS-E-It and FIM scores at admission and discharge are reported in Table 1. The RCS-E-It scores at admission had a median score of 10 (range 8–14), FIM motor 51 (range 13–88) and FIM cognitive 35 (range 5–35). At discharge the RCS-E-It scores median score was 6 (range 4–12), FIM motor was 77 (range 13–90) and FIM cognitive was 35 (range 5–35). The items RCS-C, RCS-N and RCS-M were decreased at discharge. Particularly, it is worth to remark that the decreasing of the item RCS-M at discharge was due to the progressive medical improvement of the patients so that they did no more require a high intensive specialized setting, but they could be discharged to a less intensive setting or at home. The item RCS-TD increased due the needs to include even more different therapy disciplines along the hospital stay with respect to the admission. RCS-TI and RCS-E items remained quite stable based on the single intensive care setting explored, and the same need of aids along the hospital stay. Validity A weak correlation between the RCS-E-It total score and subscales at admission and FIM score (motor and cognitive), CIRS, Braden and Morse scores was found (Table 2).
4
L. Galletti et al.
Disabil Rehabil, Early Online: 1–10
Table 1. Descriptive statistics for scores at admission and discharge. Admission (n ¼ 100)
RCS C (0–4) RCS N (0–3) RCS M (0–3) RCS TD (0–4) RCS TI (0–4) RCS E (0–2) RCS TOT (0–20) Motor FIM Cognitive FIM Total FIM
Discharge (n ¼ 100)
Wilcoxon signed rank test
Mean ± SD
Median
Range
Mean ± SD
Median
Range
z
p
1.47 ± 0.86 1.96 ± 0.24 2.02 ± 0.14 1.29 ± 0.46 3.04 ± 0.37 0.98 ± 0.14 10.75 ± 1.24 43.04 ± 19.25 28.46 ± 9.58 71.50 ± 26.84
1 2 2 1 3 1 10 51 35 86
0–3 1–3 2–3 1–2 2–4 0–1 8–14 13–88 5–35 18–115
0.66 ± 0.97 0.23 ± 0.42 0.08 ± 0.27 1.72 ± 0.78 2.90 ± 0.50 0.91 ± 0.32 6.50 ± 2.09 64.25 ± 24.24 30.01 ± 8.20 94.26 ± 31.01
0 0 0 2 3 1 6 77 35 112
0–3 0–1 0–1 1–3 0–4 0–2 4–12 13–90 5–35 18–122
8.16 9.15 9.62 5.85 2.27 2.33 8.73 8.42 4.63 8.41
50.0005 50.0005 50.0005 50.0005 0.23 0.02 50.0005 50.0005 50.0005 50.0005
Disabil Rehabil Downloaded from informahealthcare.com by Nyu Medical Center on 06/08/15 For personal use only.
Table 2. Spearman correlations between RCS scores and other indicators at admission.
RCS N RCS M RCS TD RCS TI RCS E RCS TOT FIM motor FIM cognitive CIRS Braden Morse
RCS C
RCS N
RCS M
RCS TD
RCS TI
RCS E
RCS TOT
FIM motor
FIM cognitive
CIRS
Braden
0.24* 0.17 0.37*** 0.12 0.18 0.82*** 0.79*** 0.72*** 0.55*** 0.75*** 0.22*
0.02 0.02 0.02 0.27** 0.40*** 0.34** 0.08 0.05 0.17 0.07
0.16 0.01 0.01 0.18 0.16 0.19 0.03 0.11 0.04
0.13 0.22* 0.59*** 0.33** 0.55*** 0.22* 0.51*** 0.21*
0.02 0.26** 0.01 0.14 0.01 0.15 0.09
0.18 0.24* 0.13 0.04 0.03 0.06
0.75*** 0.65*** 0.46*** 0.70*** 0.25*
0.70*** 0.62*** 0.76*** 0.26*
0.64*** 0.73*** 0.29**
0.65*** 0.12
0.34**
*p50.05, **p50.01, ***p50.001.
A negative correlation was found between the RCS-E-It total score and subscales C, N, TD and E and the FIM motor score. RCS-E-It total score and subscales C and TD were inversely correlated also to cognitive FIM. Total RCS-E-It score and subscales C and TD were correlated with number of co-morbidities (according to the CIRS scale) and also with the Morse score and the Braden score. Reliability Inter-rater agreement/reproducibility: The ICC, concerning the total RCS-E-It score, was 0.882 (p50.0005). With respect to the partial items of the scale, the classification identity rates between the two observers, Cohen’s kappa coefficient values and the p significance are shown in Table 3. While a substantial agreement was found for subscales C, M, TD and S, the agreement for TI was moderate and for N even fair. Looking at the identification rate, an agreement was, however, present, respectively, in 96% and 86% of cases. Test–retest repeatability: the ICC concerning the total RCS-EIt score was 0.93 (p50.0005) for observer 1 and 0.77 (p50.0005) for observer 2. Classification identity rates between the two observers for each observer, Cohen’s kappa coefficient values and the p significance are shown in Table 3. Agreement for subscales ranged from substantial to almost perfect for both the observers. However, observer 2 tended to attribute the same score in the two assessment sessions for most of the subscales. The lack of Cohen’s kappa coefficient values and significance for items C, N, M and E is thus attributable to the total absence of variability of the sample and the consequent impossibility to perform statistical analyses.
Table 3. Inter-rater and test–retest agreement.
ICC Cronbach alpha p RCS C K cohen Id p RCS N K cohen Id p RCS M K cohen Id p RCS TD K cohen Id p RCS TI K cohen Id p RCS E K cohen Id p
Inter-rater agreement
Test–retest agreement obs1
Test–retest agreement obs2
0.882 0.944 50.0005
0.93 0.968 50.0005
0.772 0.876 50.0005
0.69 86% 5 0,0005
0.779 90% 50.0005
1 100% 50.0005
0.267 95% 0.001
0.793 98% 50.0005
– 100% –
0.662 99% 50.0005
0.999 100% 0.01
– 100% –
0.749 85% 50.0005
0.814 89% 50.0005
0.672 92% 50.0005
0.582 96% 50.0005
0.732 0.960 50.0005
0.847 98% 50.0005
0.152 91% 0.041
0.651 96% 50.0005
– 100% –
DOI: 10.3109/09638288.2015.1024340
Disabil Rehabil Downloaded from informahealthcare.com by Nyu Medical Center on 06/08/15 For personal use only.
Discussion The present study was aimed at providing Italian translation, transcultural validation and analysis of the psychometric properties (retrospective applicability, validity and reliability) of the validated RCS-E on a sample of patients with stroke sequelae and total hip arthroplasty admitted to an Intensive Rehabilitation Ward. According to guidelines’ recommendations [20,21], the transcultural validation was successfully performed through the translation of the scale in Italian, its back translation, the revision by a committee of experts and its testing in the clinical setting to ensure that the meaning of the original version was kept also in the translation. The translation of the RCS-E 12th version presented some problems in comparing the UK rehabilitation system and the Italian system, thus requiring an adaptation of the translation. Main differences were relative to (1) the level of specialization of nurses, which does not formally exist in Italy, (2) the medical needs, which in Italy are essentially based on different services with respect to the intensity of rehabilitation required and hence (3) the intensity level of rehabilitation care that is well stated in most of the Italian regions, and determines the reimbursement from the National Health System. Intensive Rehabilitation wards provide a minimum of 3 h/day, at least 5 days a week to complex patients with higher rehabilitation needs, while Extensive Rehabilitation wards provide a rehabilitative time ranging from 1 to 3 h/day to patients who are less demanding from a rehabilitation point of view but still require hospital care. This distinction actually is not applicable to the Intensity of Treatment in the original scale: TI1 which corresponds to a ‘‘Low level – less than daily (e.g. assessment/review/maintenance/supervision) OR Group therapy only’’ does not correspond exactly in fact to the Extensive Rehabilitation in Italy, as well as TI2 that is ‘‘Moderate – daily intervention – individual sessions with one person to treat for most sessions OR very intensive Group program of 6 h/day’’ does not correspond exactly to the Intensive Rehabilitation in our country. Furthermore TI3 ‘‘High level – daily intervention with therapist PLUS assistant and/or additional group sessions’’ and TI4 ‘‘Very High level – very intensive (e.g. two trained therapists to treat, or total 1:1 therapy 430 h/week)’’ are not represented in the usual rehabilitation care in Italian National Health System. Based on these considerations, it must be taken into account that the assumptions made and the application of the RCS-E in the Italian context could inevitably lead to ceiling effects which are a product of the scoring rules, rather than the instrument itself thus impacting on comparability between countries. The RCS-E-It scale proved to be retrospectively applicable to all the clinical files examined. Although the first publication of the original RCS score in 2007 suggested that a median of 9/15 as a cut-off for complex specialised services [14], this was a cross-sectional analysis including patients at all stages of their rehabilitation programme, including those nearing discharge. Recent figures from the UK Rehabilitation Outcomes Collaborative (based on data collected 2010–2014) record a mean of 12.2 (SD 2.6) for RCS-E-v12 on admission to Level 1 Complex Specialised Services, and of 11.3 (SD 2.5) for admission to a Level 2 service (Personal communication Professor Turner-Stokes). Therefore, the mean value of RCS-E found in the present study probably corresponds to a lower intensity rehabilitation service in UK. This can be explained by the presence in the sample of patients without significant cognitive/communication deficits (the Median FIM Cognitive score is 35, i.e. the maximum score). Moreover, half of the patients were undergoing fairly simple orthopedic rehabilitation
Validation of RCS-E Italian version
5
after a hip replacement (which would be managed in a Level 3 service in the UK). Results obtained in the orthopedic sample could suggest the need for a more appropriate setting characterized by a lower rehabilitation complexity for these patients. In agreement with the findings of clinimetric properties of RCS 2 [14] and RCS-E [15], the comparison with the FIM score provided weak correlations with RCS-E-It, especially concerning the RCS-E scores related to the need for medical care (item M) and therapeutic needs (items TD + TI). As stated by TurnerStokes [15] comparison of RCS-E-It with the FIM score shows how this disability scale provides good indications on the need for basic and nursing care, but is a poor indicator of therapeutic and medical care needs. Particularly, the expected negative correlation between the RCS-E-It score and the FIM score as well as the statistically significant correlation obtained with CIRS, Braden score and Morse score confirms the validity of the scale in assessing complexity of patients from basic care, nursing, medical and therapy needs. However, as none of these are direct measures of therapy or medical needs, only a weak correlation between the RCS-E-It and these scales was found. Unfortunately, it was not possible to exactly reproduce the validation of the original scale because Northwick Park Nursing and Therapy Dependency Scales are not used in our country [14]. Anyway, within the available comparisons, we believe that RCS-E-It behaved similarly to the original, confirming its value in assessing the complexity of rehabilitation needs and its complementary role among other outcome measures. The ICC showed for the total RCS-E-It score an excellent inter-rater agreement between the two observers (0.882). This is particularly relevant because, as stated by Bland and Altman [22], while values of 0.7–0.8 are regarded as satisfactory for scales used as research tools, a minimum of 0.90 is desirable for clinical applications. A lower agreement rate was found for subscales, which ranged from fair agreement for N to substantial agreement for other subscales mostly confirming findings of original RCS-E version 2 validations [15]. Actually, looking at data, only in 5% of cases the agreement on N score was not present, confirming, however, some difficulty in identifying correctly the type (N1/N2/N3) of nursing assistance. In the test–retest agreement, the ICC was excellent for observer 1 (0.93), while a lower ICC (0.77) was found for observer 2. In this case, statistical results for subscales were affected by the absence of variation in many answers of this observer for most of the subscales. For both observers, all subscales showed an agreement ranging from substantial to almost perfect. A limitation of the present study is the small number of patients and the case mix, which differed from the validation study of the original scale, including also orthopedic patients. Considering that the clinimetric properties of RCS-E were already previously assessed by other authors [14,15], the small sample has been judged adequate for the transcultural validation. Another potential weakness of our study design was the retrospective application of the scale, which is not the way it is supposed to be used in clinical setting. This choice, however, granted us some advantages for the validation purposes, namely the fact of having complete and stable information from the clinical records for all cases, allowing us to perform retest for reliability after a considerable amount of time, thus minimising memory effect. However, it also implies some disadvantages, given that not all interventions may have been fully recorded. This could have introduced a bias, under-rating the activities and reducing the opportunity for comparison with samples where the RCS is scored in real time by clinicians. Furthermore, this additional step between the patient and the rater may affect inter-rater reliability,
6
L. Galletti et al.
and scoring discrepancies may reflect observer differences in interpretation of the clinical records, rather than differences in the use of the rating scale itself. In conclusion, the Italian validation of the RCS-E confirmed that the scale has good psychometric properties, in particular in terms of validity and repeatability. However, a particular adaptation has been required for subscales ‘‘Nursing’’, ‘‘Medical’’ and ‘‘Therapy Intensity’’ for cultural Italian health system structure differences with the UK system which could prevent comparison. Further studies, with particular attention to the adapted subscales, to homogeneous groups of pathologies, and with a prospective application of the method, will be needed to make RCS-E-It a routinely used tool to assess the appropriateness of the rehabilitation setting in Italy.
Disabil Rehabil Downloaded from informahealthcare.com by Nyu Medical Center on 06/08/15 For personal use only.
Acknowledgements The authors are grateful to Keith Smith and Brett Martyn for the back translation of the RCS-E-It scale. Professor Lynne Turner Stokes, developer of the RCS scale, is acknowledged for her invaluable help with the critical interpretation of the study results.
Declaration of interest The authors report no declaration of interest.
References 1. Castaldi S, Bevilacqua L, Arcari G, et al. How appropriate is the use of rehabilitation facilities? Assessment by an evaluation tool based on the AEP protocol. J Prev Med Hyg 2010;51:116–20. 2. World Health Organization. Appropriateness in health care services. Report on a WHO Workshop; 2000 Mar 23–25; Koblenz, Germany. 3. Berwick DM, Hackbarth AD. Eliminating waste in US health care. JAMA 2012;307:1513–16. 4. Ministero della Salute. Piano d’indirizzo per la riabilitazione – Suppl. Ord. N. 60, G.U. n . 50 del 2/3/2011. Available from: http:// www.salute.gov.it [last accessed 31 Aug 2014]. 5. Heinemann AW, Linacre JM, Wright BD, et al. Relationships between impairment and physical disability as measured by the Functional Independence Measure. Arch Phys Med Rehabil 1993; 74:566–73. 6. Mahoney FI, Barthel DW. Functional evaluation: the Barthel Index. Md State Med J 1965;14:61–5.
Disabil Rehabil, Early Online: 1–10
7. Linn BS, Linn MV, Gurel L. Cumulative illness rating scale. J Am Geriatr Soc 1968;16:622–6. 8. Guile´ R, Leux C, Paille´ C, et al. Validation of a tool assessing appropriateness of hospital days in rehabilitation centres. Int J Qual Health Care 2009;21:198–205. 9. Poulos CJ, Eagar K. Determining appropriateness for rehabilitation or other subacute care: is there a role for utilisation review? Aust New Zealand Health Policy 2007;4:1–7. 10. Poulos CJ, Magee C, Bashford G, Eagar K. Determining level of appropriateness in the patient journey from acute care to rehabilitation. BMC Health Service Res 2011;11:1–9. 11. Sutherland JM, Walker J. Challenges of rehabilitation case mix measurement in Ontario hospitals. Health Policy 2008;85:336–48. 12. Eagar K. The Australian National Sub-Acute and Non-Acute Patient casemix classification. Aust Health Rev 1999;22:180–96. 13. Turner-Stokes L, Disler R, Williams H. The Rehabilitation Complexity Scale: a simple, practical tool to identify ‘complex specialized’ services in neurological rehabilitation. Clin Med 2007; 7:593–9. 14. Turner-Stokes L, Williams H, Siegert RJ. The Rehabilitation Complexity Scale version 2: a clinimetric evaluation in patient with severe complex neurodisability. J Neurol Neurosurg Psychiatry 2010;81:146–53. 15. Turner-Stokes L, Scott H, Williams H, Siegert R. The Rehabilitation Complexity Scale – extended version: detection of patient with highly complex needs. Disabil Rehabil 2012;34:715–20. 16. Turner-Stokes L, Sutch S, Dredge R. Healthcare tariffs for specialist inpatient neurorehabilitation services: rationale and development of a UK casemix and costing methodology. Clin Rehab 2012;26: 264–79. 17. Roda F, Agosti M, Corradini E, et al. Cross-cultural adaptation and preliminary test–retest reliability of the Italian version of the Complexity Rehabilitation Scale-Extended (13th Version). Eur J Phys Rehabil Med 2014 Mar 04. [Epub ahead of print]. 18. Bergstrom N, Braden BJ, Laguzza A, Holman V. The Braden Scale for predicting pressure sore risk. Nurs Res 1987;36:205–10. 19. Morse JM, Morse RM, Tylko SJ. Development of a scale to identify the fall-prone patient. Can J Aging 1989;8:366–7. 20. Beaton D, Bombardier C, Guilleman F, et al. Recommendations for the cross-cultural adaptation of health status measures. Rosemont (IL): American Academy of Orthopaedic Surgeons, Institute for Work and Health; 1998. 21. Beaton DE, Bombardier C, Guillemin F, et al. Guidelines for the process of cross-cultural adaptation of self-report measures. Spine 2000;25:3186–91. 22. Bland JM, Altman DG. Statistics notes: Cronbach’s alpha. BMJ 1997;314:572.
Disabil Rehabil Downloaded from informahealthcare.com by Nyu Medical Center on 06/08/15 For personal use only.
DOI: 10.3109/09638288.2015.1024340
Appendix 1.
Validation of RCS-E Italian version 7
Disabil Rehabil Downloaded from informahealthcare.com by Nyu Medical Center on 06/08/15 For personal use only.
8 L. Galletti et al. Disabil Rehabil, Early Online: 1–10
Disabil Rehabil Downloaded from informahealthcare.com by Nyu Medical Center on 06/08/15 For personal use only.
DOI: 10.3109/09638288.2015.1024340
Validation of RCS-E Italian version 9
Disabil Rehabil Downloaded from informahealthcare.com by Nyu Medical Center on 06/08/15 For personal use only.
10 L. Galletti et al. Disabil Rehabil, Early Online: 1–10
