Authors: Marco Streibelt, PhD Matthias Bethge, PhD
Predictive Modelling
Affiliations: From the Department of Rehabilitation, German Federal Pension Insurance, Berlin (MS); and Institute of Social Medicine and Epidemiology, University of Lu¨beck, Lu¨beck, Germany (MB).
Correspondence: All correspondence and requests for reprints should be addressed to: Marco Streibelt, PhD, Department of Rehabilitation, German Federal Pension Insurance, Hohenzollerndamm 47, D-10704 Berlin, Germany.
ORIGINAL RESEARCH ARTICLE
Prospective Cohort Analysis of the Predictive Validity of a Screening Instrument for Severe Restrictions of Work Ability in Patients with Musculoskeletal Disorders
Disclosures: Dr. Streibelt works at the GFPI, where the screening instrument, SIMBO, is used in the access management to rehabilitation services. Financial disclosure statements have been obtained, and no conflicts of interest have been reported by the authors or by any individuals in control of the content of this article.
0894-9115/15/9408-0617 American Journal of Physical Medicine & Rehabilitation Copyright * 2014 Wolters Kluwer Health, Inc. All rights reserved. DOI: 10.1097/PHM.0000000000000220
ABSTRACT Streibelt M, Bethge M: Prospective cohort analysis of the predictive validity of a screening instrument for severe restrictions of work ability in patients with musculoskeletal disorders. Am J Phys Med Rehabil 2015;94:617Y626.
Objective: The aim of this study was to determine whether the ScreeningInstrument zur Feststellung des Bedarfs an medizinisch-beruflich orientierter Rehabilitation (SIMBO) screening instrument can identify persons with a high work disability risk. Design: Patients with chronic musculoskeletal disorders and participation in a rehabilitation program were included in the analysis. Data were collected by questionnaires at admission and at the 6-mo follow-up. Failed return to work (RTW; i.e., unemployment or active employment with sick leave of 912 wks in the follow-up) was the primary outcome. Additional outcome data were obtained from the physician’s discharge form (e.g., assessment of work ability). Receiver operating characteristic analyses and logistic regression models were used to analyze the data.
Results: Valid data were available for 1755 participants. Of these, 25% reported failed RTW. The area under the curve of the SIMBO score predicting failed RTW was 0.81 (0.79Y0.83). The optimal cutoff to identify failed RTW was 23 points (of 100). The odds of failed RTW, unemployment, and sick leave of more than 12 wks were increased 12, 11, and 10 times for the persons with a SIMBO score of 23 points or greater compared with the persons with lower SIMBO scores. The odds of reduced work time capacity were increased 21 times.
Conclusions: The SIMBO predicts work disability in patients with chronic musculoskeletal disorders. Further research should focus on its predictive validity in patients with other disorders and within other rehabilitation systems. Key Words: Multidisciplinary Rehabilitation, Musculoskeletal Disorders, Work Disability, Predictive Validity
www.ajpmr.com
Predictive Validity of the SIMBO Screening Copyright © 2015 Wolters Kluwer Health, Inc. All rights reserved.
617
R
ehabilitation services are provided to prevent work disability and early retirement as a result of chronic musculoskeletal disorders (CMSDs). However, the effectiveness of these rehabilitation measures depends on their design and content. In the case of CMSDs, multimodality is discussed as an important success factor.1 Moreover, several studies stressed that the relationship between the employee’s work environment and demands determines the success of rehabilitation measures.2 In their updated Cochrane review, Schaafsma et al.3 concluded that rehabilitation is more effective if interventions are executed at the workplace or include a workplace visit. In addition, risk-adjusted treatment programs, that is, problem-treatment pairs, are the key for an effective rehabilitation service management. Several studies have shown this for a range of different rehabilitation settings. Physical conditioning is not effective in patients with acute back pain3,4; comprehensive medical rehabilitation is not more effective as usual care in patients who do not have severe chronic diseases.5,6 Furthermore, some studies have shown that a higher risk for permanent work disability strengthens the effectiveness of intensive rehabilitation measures. A Dutch study showed that a workplace intervention increased the return to work (RTW) rate only in people with previous episodes of long-term sick leave.7 A subgroup analysis of a Danish randomized controlled trial revealed that a multidisciplinary intervention was associated with faster RTW only in patients with low job satisfaction, who had no influence on work planning, and who felt at risk for losing their jobs because of their sick leave.8 In addition, in a pilot study from Canada, an intensified work-related multidisciplinary intervention was compared with conventional case management. A stratified analysis according to the risk for work disability revealed that no effect on RTW was observed in the group with moderate risk (72% vs. 80%) but that the intensified rehabilitation program showed a significant effect in the group of high-risk people (71% vs. 29%).9 It seems obvious that intensified and workrelated rehabilitation measures can yield meaningful results for patients having a high risk for permanent work disability that are not realized in medium- or lower-risk groups. In Germany, such intensified work-related medical rehabilitation programs have been implemented during the last 10 yrs.10 These programs comprise additional measures such as functional capacity evaluation, functional capacity training, work-related psychological groups, and in-
618
Streibelt and Bethge
tensified social counseling; have a stronger focus on RTW than do conventional German rehabilitation programs; and were designed for patients with strongly limited work ability (e.g., long-term sick leave) and poor self-reported RTW prognosis. A set of randomized controlled trials showed concordantly that increased RTW rates as a result of workrelated multidisciplinary rehabilitation (MR) were measured only in patients with severe restrictions of work ability.11,12 Patients with low or moderate work-related restrictions did not experience any additional benefit from work-related MR.13 Therefore, assessment of the work disability risk and allocation to rehabilitation programs (work-related or conventional MR) according to the identified risk level are of great importance for an effective rehabilitation process.14 In this context, the use of valid screening instruments that measure self-rated work ability could be an appropriate and efficient strategy.15,16 In Germany, a short screening instrument for the identification of severe restrictions of work ability was developed for allocation to work-related or conventional MR (German: Screening-Instrument zur Feststellung des Bedarfs an medizinisch-beruflich orientierter Rehabilitation [SIMBO]). Agreement between the need assessment based on the screening and the need rating of rehabilitation physicians was approximately 80% in one study.17 Another study in patients with several chronic diseases (musculoskeletal, mental, and internal disorders) showed the predictive validity of this screening for RTW.18 In the present article, further results on the predictive validity of the screening for critical RTW follow-up events are reported using a combined meta-sample from several cohorts of rehabilitation patients with CMSDs.
METHODS Setting and Participants This analysis included patient samples from three former prospective cohort studies that assessed similar data and was initially performed to investigate the quality and effects of German MR. One study recruited patients in two rehabilitation centers, whereas the other two studies were singlecenter studies. The sample consisted of patients with different CMSDs who received inpatient rehabilitation on behalf of the German Pension Insurance between 2004 and 2008.17,18 The German Pension Insurance is responsible for rehabilitation service of persons with chronic diseases and a risk for permanent work disability
Am. J. Phys. Med. Rehabil. & Vol. 94, No. 8, August 2015 Copyright © 2015 Wolters Kluwer Health, Inc. All rights reserved.
not caused by occupational accidents. Rehabilitation measures need a format request of the patient in need. An initial need assessment of German Pension Insurance physicians leads then to approval or refusal of a request and is usually done on the basis of the patient’s self-reports and a supporting report of the patient’s general practitioner. After approval of the request, the German Pension Insurance allocated the patients to one of the participating rehabilitation centers. The inpatient rehabilitation followed a multidisciplinary approach. The programs lasted 3Y4 wks (approximately 50Y70 hrs in total) and included exercise, physical therapy, social counseling, patient education (e.g., pain and stress management), as well as passive therapies such as massages and therapeutic baths.19 The multiprofessional team involved physicians, exercise trainers, physical and occupational therapists, psychologists, as well as social workers. Recruitment was usually carried out by the attending physician. After informed consent was obtained, the patients were given a questionnaire at admission. After 6 mos, the researchers sent a postal questionnaire to the participants to collect followup data. In addition, the questionnaire data were linked to the physician-reported data obtained from the standardized rehabilitation discharge form.20 Participants without active employment (e.g., housewives) and those receiving a retirement benefit were excluded. Furthermore, participants with a missing value in the interesting variables were excluded.
The SIMBO The SIMBO score is calculated as the weighted sum of seven single items. The total score ranges from 0 to 100 points (Fig. 1). Higher values indicate an increased risk for permanent work disability and therefore a need for additional work-related interventions according to the recently published guideline for work-related MR.21,22 The single items (unemployment, current sick leave episode, sickness absence of 96 mos during the past year, subjective work disability [0Y10] 9 7 points, expectation of not being able to work at all, motivation for work-related interventions [1Y5] 9 3 points, and age G 46 yrs) are dichotomous criteria that were identified as risk factors of failed RTW as a result of CMSD in former cohort studies and structured reviews23Y26 (summarized in Streibelt and Bethge18). The weights of the single items were empirically derived from prediction models.17 www.ajpmr.com
Outcome Measures and Covariates Critical RTW Events During the 6-mo Follow-up Period The main outcome was failed RTW.16 RTW is, by law, the overall goal of rehabilitation services in Germany. Because sick leave duration is a strong predictor of disability pension, RTW evaluation needs also to consider follow-up sick leave duration. Therefore, RTW was defined as active employment with a maximum of 12 wks of sick leave during the 6-mo follow-up period, whereas failed RTW meant unemployment or active employment but sick leave duration of more than 12 wks during the 6-mo follow-up period. In addition, unemployment vs. employment and employment with sick leave of more than 12 wks vs. employment with sick leave of 12 wks or less were considered as additional outcomes.
Physician-Reported Work Ability Assessment at Discharge The physician’s assessment of the patient’s work ability was obtained from the standardized discharge form. These assessments include recommendations regarding vocational rehabilitation (VR), that is, a qualification measure if a former job can no longer be performed, and the evaluation of daily work time capacity in the current job (96 hrs per day vs. e6 hrs per day). This evaluation is the foundation of further decisions concerning disability pension claims.
Sociodemographics and Covariates Age, sex, and self-rated health (visual analog scale, 0Y100 points) were assessed as covariates. To characterize the sample, socioeconomic status (low, middle, high)27 was also assessed, and the primary treatment diagnosis was obtained from the standardized discharge form.
Analysis Descriptive statistics were used to present the baseline sample characteristics, including the SIMBO score and single SIMBO items as well as the prevalence of the outcome measures. To analyze the predictive characteristics of the SIMBO score for each outcome measure, a receiver operating characteristic curve was constructed, and the area under the curve (AUC) was determined.28 The AUC represents the probability that a randomly selected person with an adverse outcome (e.g., failed RTW) will have a higher SIMBO score than a randomly selected person without this outcome. Values of greater than 0.5 indicate a better prediction than chance. Sensitivity, specificity, and percentage of agreement were calculated using each SIMBO score as a cutoff value to classify persons Predictive Validity of the SIMBO Screening
Copyright © 2015 Wolters Kluwer Health, Inc. All rights reserved.
619
FIGURE 1 Screening for the identification of severe restrictions of work ability (SIMBO) including a calculation guideline.
as either high-risk or low-risk patients. The optimal SIMBO cutoff is then the one that maximizes the Youden index (J), which is defined as sensitivity + specificity j 1.29 In addition, the positive and negative predictive value were calculated for the optimal cutoff. Finally, for each outcome measure, random intercept logistic regression models were calculated to estimate how much higher the odds of an adverse outcome are for the patients with a high SIMBO score, that is, greater than or equal to the optimal cutoff value, compared with the persons with a low SIMBO score, that is, less than the optimal cutoff value. Effect estimates were adjusted for age, sex, and self-rated health. A random intercept was used to account for the clustered data within the study samples. Statistical differences were regarded as significant if the two-sided P value of the test was less than 0.05. All calculations were performed using Stata 10.0.
620
Streibelt and Bethge
RESULTS Sample The four samples comprised a total of 2545 participants. Of these, 1985 (78%) responded to the 6-mo follow-up questionnaire. After the exclusion of participants without active employment and those receiving a retirement benefit (n = 90) as well as participants with a missing SIMBO score or a missing outcome score (n = 140), the remaining sample consisted of 1755 participants. Data from the standardized discharge form were available for only three of the four rehabilitation centers involved (n = 1425). A total of 69.8% of the patients had dorsopathies (International Classification of Diseases, 10th Revision: M40YM55), followed by arthropathies (M00YM25) and soft tissue disorders (M60YM79). The mean age was 47.7 yrs; 34.6% were women. The sample had
Am. J. Phys. Med. Rehabil. & Vol. 94, No. 8, August 2015 Copyright © 2015 Wolters Kluwer Health, Inc. All rights reserved.
a mean (SD) SIMBO score of 24 (23.6) points. The distribution of the SIMBO scores was right skewed. The mass of the patients had low scores, giving further support for the construct validity of the SIMBO scale. The prevalence of the single SIMBO criterion ranged from 8.6% (unemployment) to 65.5% (motivation for work-related MR). The baseline characteristics are reported in Table 1. The physicians stated a need for further VR after MR for 14.7% of the patients. A total of 11.7% were evaluated as able to work for not more than 6 hrs per day. Six months after discharge of MR, 25% of the participants failed RTW; 17.3% were unemployed and 13.6% reported sick leave of more than 12 wks during the 6-mo follow-up period.
SIMBO Score and Failed RTW: The Optimal Cutoff The AUC predicting failed RTW was 0.82 (95% confidence interval [CI], 0.80Y0.84; Table 2 and Fig. 2) and confirmed the predictive validity of the SIMBO score. The AUCs for predicting unemployment (AUC, 0.81; 95% CI, 0.79Y0.83) and long-term sick leave (AUC, 0.82; 95% CI, 0.80Y0.83) were similar. Work
time capacity was also well predicted by the SIMBO score (AUC, 0.89; 95% CI, 0.88Y0.91). The prediction of recommended VR was somewhat worse (AUC, 0.78; 95% CI, 0.76Y0.80) (Fig. 3). The optimal cutoff of the SIMBO score predicting failed RTW was 23 points. Given this cutoff, sensitivity and specificity were 80% and 76%, respectively (Table 3). A total of 77% of the patients were correctly classified. Sensitivity decreased as the cutoff increased and vice versa. Between 15 and 39 points of the SIMBO score, the Youden index was 0.50 or greater, meaning that the mean of sensitivity and specificity was greater than or equal to 75%. Fifty-two percent of those patients tested positive on the SIMBO (Q23 points) failed RTW after rehabilitation (positive predictive value). Ninety-two percent of those patients, who were identified as low-risk patients (G23 points), successfully returned to work after rehabilitation (negative predictive value).
Predicting Critical RTW Events at Follow-up Using the Optimal SIMBO Cutoff The associations of the baseline SIMBO score and follow-up outcomes are presented in Table 4.
TABLE 1 Sample characteristics Characteristic Baseline Age, mean (SD), yrs Sex: female Diagnosis (ICD-10) Arthropathies (M00YM25) Dorsopathies (M40YM54) Soft tissue disorders (M60YM79) Other Social status Low Middle High Self-rated health, mean (SD) SIMBO score, mean (SD) Unemployed Sick listed Sick leave (96 mos) Work disability (97 points) Negative subjective RTW prognosis Motivation for work-related MR Age (G46 yrs) Outcome measures Discharge form Recommended VR Work time capacity in current job (e6 hrs per day) Follow-up Unemployment Sick leave (912 wks) Failed RTW
n
Value
1755 1755
47.7 (8.7) 34.6
168 1225 70 62 1734 302 1182 250 1755 1755 1755
11.0 80.3 4.6 4.1 17.4 68.2 14.4 52.4 (18.0) 24.0 (23.6) 8.6 31.6 12.8 31.2 11.1 65.5 34.8
1417 1412
14.7 11.7
1755 1755 1755
17.3 13.6 25.0
Values are mean (SD) or percentage.
www.ajpmr.com
Predictive Validity of the SIMBO Screening Copyright © 2015 Wolters Kluwer Health, Inc. All rights reserved.
621
TABLE 2 AUC for predicting work-related outcomes using the SIMBO score Outcome Measures Follow-up Failed RTW Unemployment Sick leave (912 wks) Discharge form Work time capacity in current job (e6 hrs per day) Recommended VR
The odds of failed RTW in persons with a baseline SIMBO score of 23 points or greater were 12 times higher than those with a lower score. Moreover, the odds of unemployment and long-term sick leave in this group were 11 times and 10 times higher, respectively. Accordingly, the adjusted prevalence rates of failed RTW events were 0.28Y0.52 for the persons with higher SIMBO scores. Failed RTW events were rare for the persons with low SIMBO scores (0.04Y0.08).
Predicting Physician-Reported Work Ability Assessment at Discharge Using the Optimal SIMBO Cutoff The associations between the baseline SIMBO score and physician-reported work ability assessment at discharge are also reported in Table 4. The odds of decreased capacity in the current job were 20.5 times higher in the patients with SIMBO scores of 23 points or greater. Hardly anyone of the patients with a low SIMBO score was identified as having decreased capacity in their current job. The
n
AUC
95% CI
1755 1755 1755
0.822 0.809 0.815
0.804Y0.840 0.789Y0.827 0.796Y0.833
1412 1417
0.892 0.781
0.875Y0.908 0.758Y0.802
odds of a recommended VR program were also higher for the patients with higher SIMBO scores (odds ratio, 7.2).
Association with Sociodemographic Characteristics Figure 4 shows that the prevalence rate of severe restrictions of work ability is strongly associated with sociodemographic characteristics. Persons with a low social status (i.e., low education or low income) have a higher risk for severe restrictions of work ability. With increasing age, the risk for severe restrictions of work ability decreases steadily until the age of 60 yrs.
DISCUSSION The aim of the present study was to clarify the relevance of the SIMBO screening instrument for identifying patients with a high risk for work disability after a multimodal rehabilitation program. The analyses of this study demonstrate the ability of the SIMBO to predict critical RTW events such as unemployment or long-term sick leave at follow-up
FIGURE 2 Receiver operating characteristic curves for predicting work-related outcomes at follow-up using the baseline SIMBO score.
622
Streibelt and Bethge
Am. J. Phys. Med. Rehabil. & Vol. 94, No. 8, August 2015 Copyright © 2015 Wolters Kluwer Health, Inc. All rights reserved.
FIGURE 3 Receiver operating characteristic curves for predicting physician-reported work ability assessment using the baseline SIMBO score.
as well as physician-reported assessments of work ability. The SIMBO is therefore a useful tool for identifying persons with a high risk for work disability before starting the rehabilitation process. Further studies have shown that administrative or medical decisions usually have low validity when they are based on insufficient information.14,30,31 In such cases, a Bpositive selection[ or Bcreaming[ can be observed, meaning that the physicians or case managers focus on the attributed goal instead of the achieved changes when deciding on access to particular services.32 In this context, screening instruments provide condensed and validated information that is independent of expectations, experiences, and subjective theories. In Germany, the use of screening instruments for access to rehabilitation services is discussed for approximately 10 yrs. Numerous studies in the German rehabilitation system have shown that the quality of decisions concerning the need for rehabilitation33 and access to specialized rehabilitation services34,35 could be improved by using screening
instruments. If the effects of an intervention are proven only for certain patients, proper identification of the target group is needed to maximize outcomes with respect to the reality of budget constraints in the area of rehabilitation. Although there are only few studies that deal with the relevance of screening-based risk profile selection, the authors of a recent review concerning the effectiveness of RTW treatments for sick-listed patients concluded that Bit may be advisable in composing an effective treatment program to take account of the duration of the sick leave before starting treatment or making a return to work prognosis.[36 For persons with a need for rehabilitation, the SIMBO is particularly useful for identifying subgroups with a high risk for permanent work disability as a result of chronic diseases. This offers the opportunity to assign these patients to intensified work-related rehabilitation measures. The prevalence rate of severe restrictions of work ability was 38% in this sample. However, the observed prevalence is dependent on the cutoff
TABLE 3 Sensitivity, specificity, PoA, and Youden index for selected cutoff values of the SIMBO score predicting failed RTW Cutoff Value
Sensitivity, %
Specificity, %
PoA, %
J
87.2 80.6 79.5 73.5 63.9 48.0
54.2 72.7 75.9 79.3 85.3 92.5
62.5 74.6 76.8 77.8 79.9 81.4
0.41 0.53 0.55 0.53 0.49 0.40
10 20 23 30 40 50 PoA, percentage of agreement.
www.ajpmr.com
Predictive Validity of the SIMBO Screening Copyright © 2015 Wolters Kluwer Health, Inc. All rights reserved.
623
TABLE 4 Predicting work-related outcomes: SIMBO score of 23 points or greater vs. SIMBO score of less than 23 points EMMoutcome Outcome Measures Follow-up Failed RTW Unemployment Sick leave (912 wks) Discharge form Work time capacity in current job (e6 hrs per day) Recommended VR
n
OR
95% CI
SIMBO G 23
SIMBO Q 23
1703 1703 1703
11.83 10.59 10.19
8.92Y15.67 7.62Y14.71 6.97Y14.91
0.08 0.05 0.04
0.52 0.37 0.28
1368 1373
20.54 7.19
11.06Y38.14 4.87Y10.62
0.02 0.05
0.27 0.29
All estimates are adjusted for age, sex, and subjective health. Model estimates were used to predict the means of the outcome criteria with balanced factor variables and covariates at their means. EMM, estimated marginal means for the predicted probability of the outcome; OR, odds ratio.
definition. Besides the recommended cutoff value of 23 points, different cutoff values between 15 and 40 points had also acceptable prognostic relevance. This demonstrates the high flexibility of the SIMBO. There is a high association of the prevalence with demographic characteristics. Persons with a low social status have a higher risk for severe restrictions of work ability. Moreover, older persons had a lower risk. The authors assume that this latter association reflects different access patterns to rehabilitation services in Germany. Indeed, the utilization rate of younger patients is rather low, but these patients have very strong limitations. This study has some limitations that need to be considered. First, the merged samples came from different studies, and the data of the original studies
were not collected specifically for this study. Furthermore, the critical RTW events were assessed by self-report. These limitations are balanced by the following strengths: First, the authors used a fairly large sample recruited in several centers. Second, the validity of this study’s findings was improved by a subsidiary analysis with physician-reported outcomes.
CONCLUSIONS Overall, the screening instrument SIMBO predicts work disability after a rehabilitation program in patients with CMSDs. Consequently, the biggest provider of rehabilitation services in Germany, the German Federal Pension Insurance, decided to implement the SIMBO in its routine of rehabilitation
FIGURE 4 Prevalence of SIMBO score of 23 points or greater in several subgroups.
624
Streibelt and Bethge
Am. J. Phys. Med. Rehabil. & Vol. 94, No. 8, August 2015 Copyright © 2015 Wolters Kluwer Health, Inc. All rights reserved.
management of persons with severe restrictions of work ability. Further research should focus on the validity of the instrument in different rehabilitation systems.
REFERENCES 1. Hoefsmit N, Houkes I, Nijhuis FJ: Intervention characteristics that facilitate return to work after sickness absence: A systematic literature review. J Occup Rehabil 2012;22:462Y77 2. Norlund A, Ropponen A, Alexanderson K: Multidisciplinary interventions: Review of studies of return to work after rehabilitation for low back pain. J Rehabil Med 2009;41:115Y21 3. Schaafsma FG, Whelan K, van der Beek AJ, et al: Physical conditioning as part of a return to work strategy to reduce sickness absence for workers with back pain. Cochrane Database Syst Rev 2013;8:CD001822 4. Schaafsma F, Schonstein E, Whelan KM, et al: Physical conditioning programs for improving work outcomes in workers with back pain. Cochrane Database Syst Rev 2010;1:CD001822 5. Saltychev M, Laimi K, El-Metwally A, et al: Effectiveness of multidisciplinary primary prevention in decreasing the risk of work disability in a low-risk population. Scand J Work Environ Health 2012;38:27Y37 6. Saltychev M, Laimi K, Oksanen T, et al: Does perceived work ability improve after a multidisciplinary preventive program in a population with no severe medical problems? The Finnish Public Sector Study. Scand J Work Environ Health 2013;39:57Y65 7. Steenstra IA, Knol DL, Bongers PM, et al: What works best for whom? An exploratory, subgroup analysis in a randomized, controlled trial on the effectiveness of a workplace intervention in low back pain patients on return to work. Spine (Phila Pa 1976) 2009;34:1243Y9 8. Stapelfeldt CM, Christiansen DH, Jensen OK, et al: Subgroup analyses on return to work in sick-listed employees with low back pain in a randomised trial comparing brief and multidisciplinary intervention. BMC Musculoskelet Disord 2011;12:112 9. Schultz IZ, Crook J, Berkowitz J, et al: A prospective study of the effectiveness of early intervention with high-risk back-injured workersVA pilot study. J Occup Rehabil 2008;18:140Y51
13. Streibelt M, Hansmeier T, Mu¨ller-Fahrnow W: Effects of work-related medical rehabilitation in patients with musculoskeletal disorders [in German]. Rehabilitation (Stuttg) 2006;45:161Y71 14. Saltychev M, Laimi K, Oksanen T, et al: Predictive factors of future participation in rehabilitation in the working population: The Finnish public sector study. J Rehabil Med 2011;43:404Y10 15. Bethge M, Radoschewski FM, Gutenbrunner C: The Work Ability Index as a screening tool to identify the need for rehabilitation: Longitudinal findings from the Second German Sociomedical Panel of Employees. J Rehabil Med 2012;44:980Y7 16. Bethge M, Gutenbrunner C, Neuderth S: Work Ability Index predicts application for disability pension after work-related medical rehabilitation for chronic back pain. Arch Phys Med Rehabil 2013;94:2262Y8 17. Streibelt M: Activity and Participation [Effectiveness of Work-Related Interventions in the Medical Rehabilitation on Behalf of the Pension Insurance] [in German]. Lengerich, Germany, Pabst, 2007 18. Streibelt M, Bethge M: Return to Work. Identification of Severe Restrictions of Work Ability in the Field of Medical Rehabilitation [in German]. Lengerich, Germany, Pabst, 2009 19. Deutsche Rentenversicherung: Therapeutical Standards BChronic Back Pain[. Guideline for the Medical Rehabilitation on Behalf of the German Pension Insurance [in German]. Berlin, Germany, Deutsche Rentenversicherung Bund, 2009 20. Deutsche Rentenversicherung Bund: The Physician’s Rehabilitation Discharge Report. Guideline for a Uniform Discharge Report in the Medical Rehabilitation on Behalf of the German Pension Insurance [in German]. Berlin, Germany, Deutsche Rentenversicherung Bund, 2009 21. Streibelt M, Buschmann-Steinhage R: A profile of requirements for the performance of work related medical rehabilitation from the perspective of the statutory pension insurance [in German]. Rehabilitation (Stuttg) 2011;50:160Y7 22. Deutsche Rentenversicherung Bund: Guideline for the Performance of the Work-Related Medical Rehabilitation (WMR) on Behalf of the German Pension Insurance [in German]. Berlin, Germany, Deutsche Rentenversicherung Bund, 2012
10. Lukasczik M, Wolf HD, Gerlich C, et al: Current state of vocationally oriented medical rehabilitationVA German perspective. Disabil Rehabil 2011;33:2646Y55
23. Kuijer W, Groothoff JW, Brouwer S, et al: Prediction of sickness absence in patients with chronic low back pain: A systematic review. J Occup Rehabil 2006;16:439Y67
11. Bethge M, Herbold D, Trowitzsch L, et al: Work status and health-related quality of life following multimodal work hardening: A cluster randomised trial. J Back Musculoskelet Rehabil 2011;24:161Y72
24. Steenstra IA, Verbeek JH, Heymans MW, et al: Prognostic factors for duration of sick leave in patients sick listed with acute low back pain: A systematic review of the literature. Occup Environ Med 2005;62:851Y60
12. Streibelt M, Bethge M: Effects of intensified workrelated multidisciplinary rehabilitation on occupational participation: A randomized-controlled trial in patients with chronic musculoskeletal disorders. Int J Rehabil Res 2014;37:61Y6
25. Mau M, Merkesdal S, Busche T, et al: Prognosis of labour force participation after multidisciplinary outpatient and inpatient rehabilitation for chronic back pain [in German]. Rehabilitation (Stuttg) 2002;41: 160Y6
www.ajpmr.com
Predictive Validity of the SIMBO Screening Copyright © 2015 Wolters Kluwer Health, Inc. All rights reserved.
625
26. Muche R, Ro¨sch M, Flierl S, et al: Development and validation of a prognosis model for predicting work capacity after rehabilitation based on routinely collected parameters [in German]. Rehabilitation (Stuttg) 2000;39:262Y7 27. Winkler J: Measurement of a social status index in the health surveys of the DHP, in Ahrens W, Bellach B, Jo¨ckel KH (eds): Measurement of Sociodemographic Parameters in the Epidemiology [in German]. Berlin, Germany, Robert-Koch-Institut, 1998, pp. 69Y84 28. Hanley JA, McNeil BJ: The meaning and use of the area under a receiver operating characteristic (ROC) curve. Radiology 1982;143:29Y36 29. Youden WJ: Index for rating diagnostic tests. Cancer 1950;3:32Y5 30. Kobelt A, Grosch E, Wasmus A, et al: Is it possible to predict approval of medical rehabilitation by the extent of fatigue and subjective need for rehabilitation? Development, results and acceptance of a short screening [in German]. Rehabilitation (Stuttg) 2007; 46:33Y40 31. Schwarz S, Mangels M, Sohr G, et al: The assignment of orthopaedic rehabilitants to the traditional or the behaviour-medical rehabilitation approachVIs
626
Streibelt and Bethge
it clinically relevant? [in German]. Rehabilitation (Stuttg) 2007;46:323Y32 32. Bell SH, Orr LL: Screening (and creaming?) applicants to job training programs: The AFDC homemaker-home health aide demonstrations. Labour Econ 2002;9:2, 279Y301 33. Meng K, Holderied A, Vogel H: Rehabilitation need in sociomedical assessmentVDevelopment and evaluation of an algorithm of decision-making [in German]. Rehabilitation (Stuttg) 2007;46:41Y9 34. Hu¨ppe A, Parow D, Raspe H: Effectiveness and benefit of a screening procedure to identify patients with diabetes mellitus type 2 in need of rehabilitation: A randomised, controlled trial among members of the Hamburg Mu¨nchener Health Insurance [in German]. Gesundheitswesen 2008;70:590Y9 35. Bethge M, Lo¨ffler S, Schwarz B, et al: Specific workrelated problems: Do they matter in access to workrelated medical rehabilitation? [in German]. Rehabilitation (Stuttg) 2014;53:49Y55 36. Meijer EM, Sluiter JK, Frings-Dresen MH: Evaluation of effective return-to-work treatment programs for sick-listed patients with non-specific musculoskeletal complaints: A systematic review. Int Arch Occup Environ Health 2005;78:523Y32
Am. J. Phys. Med. Rehabil. & Vol. 94, No. 8, August 2015 Copyright © 2015 Wolters Kluwer Health, Inc. All rights reserved.
Predictive Modelling
Affiliations: From the Department of Rehabilitation, German Federal Pension Insurance, Berlin (MS); and Institute of Social Medicine and Epidemiology, University of Lu¨beck, Lu¨beck, Germany (MB).
Correspondence: All correspondence and requests for reprints should be addressed to: Marco Streibelt, PhD, Department of Rehabilitation, German Federal Pension Insurance, Hohenzollerndamm 47, D-10704 Berlin, Germany.
ORIGINAL RESEARCH ARTICLE
Prospective Cohort Analysis of the Predictive Validity of a Screening Instrument for Severe Restrictions of Work Ability in Patients with Musculoskeletal Disorders
Disclosures: Dr. Streibelt works at the GFPI, where the screening instrument, SIMBO, is used in the access management to rehabilitation services. Financial disclosure statements have been obtained, and no conflicts of interest have been reported by the authors or by any individuals in control of the content of this article.
0894-9115/15/9408-0617 American Journal of Physical Medicine & Rehabilitation Copyright * 2014 Wolters Kluwer Health, Inc. All rights reserved. DOI: 10.1097/PHM.0000000000000220
ABSTRACT Streibelt M, Bethge M: Prospective cohort analysis of the predictive validity of a screening instrument for severe restrictions of work ability in patients with musculoskeletal disorders. Am J Phys Med Rehabil 2015;94:617Y626.
Objective: The aim of this study was to determine whether the ScreeningInstrument zur Feststellung des Bedarfs an medizinisch-beruflich orientierter Rehabilitation (SIMBO) screening instrument can identify persons with a high work disability risk. Design: Patients with chronic musculoskeletal disorders and participation in a rehabilitation program were included in the analysis. Data were collected by questionnaires at admission and at the 6-mo follow-up. Failed return to work (RTW; i.e., unemployment or active employment with sick leave of 912 wks in the follow-up) was the primary outcome. Additional outcome data were obtained from the physician’s discharge form (e.g., assessment of work ability). Receiver operating characteristic analyses and logistic regression models were used to analyze the data.
Results: Valid data were available for 1755 participants. Of these, 25% reported failed RTW. The area under the curve of the SIMBO score predicting failed RTW was 0.81 (0.79Y0.83). The optimal cutoff to identify failed RTW was 23 points (of 100). The odds of failed RTW, unemployment, and sick leave of more than 12 wks were increased 12, 11, and 10 times for the persons with a SIMBO score of 23 points or greater compared with the persons with lower SIMBO scores. The odds of reduced work time capacity were increased 21 times.
Conclusions: The SIMBO predicts work disability in patients with chronic musculoskeletal disorders. Further research should focus on its predictive validity in patients with other disorders and within other rehabilitation systems. Key Words: Multidisciplinary Rehabilitation, Musculoskeletal Disorders, Work Disability, Predictive Validity
www.ajpmr.com
Predictive Validity of the SIMBO Screening Copyright © 2015 Wolters Kluwer Health, Inc. All rights reserved.
617
R
ehabilitation services are provided to prevent work disability and early retirement as a result of chronic musculoskeletal disorders (CMSDs). However, the effectiveness of these rehabilitation measures depends on their design and content. In the case of CMSDs, multimodality is discussed as an important success factor.1 Moreover, several studies stressed that the relationship between the employee’s work environment and demands determines the success of rehabilitation measures.2 In their updated Cochrane review, Schaafsma et al.3 concluded that rehabilitation is more effective if interventions are executed at the workplace or include a workplace visit. In addition, risk-adjusted treatment programs, that is, problem-treatment pairs, are the key for an effective rehabilitation service management. Several studies have shown this for a range of different rehabilitation settings. Physical conditioning is not effective in patients with acute back pain3,4; comprehensive medical rehabilitation is not more effective as usual care in patients who do not have severe chronic diseases.5,6 Furthermore, some studies have shown that a higher risk for permanent work disability strengthens the effectiveness of intensive rehabilitation measures. A Dutch study showed that a workplace intervention increased the return to work (RTW) rate only in people with previous episodes of long-term sick leave.7 A subgroup analysis of a Danish randomized controlled trial revealed that a multidisciplinary intervention was associated with faster RTW only in patients with low job satisfaction, who had no influence on work planning, and who felt at risk for losing their jobs because of their sick leave.8 In addition, in a pilot study from Canada, an intensified work-related multidisciplinary intervention was compared with conventional case management. A stratified analysis according to the risk for work disability revealed that no effect on RTW was observed in the group with moderate risk (72% vs. 80%) but that the intensified rehabilitation program showed a significant effect in the group of high-risk people (71% vs. 29%).9 It seems obvious that intensified and workrelated rehabilitation measures can yield meaningful results for patients having a high risk for permanent work disability that are not realized in medium- or lower-risk groups. In Germany, such intensified work-related medical rehabilitation programs have been implemented during the last 10 yrs.10 These programs comprise additional measures such as functional capacity evaluation, functional capacity training, work-related psychological groups, and in-
618
Streibelt and Bethge
tensified social counseling; have a stronger focus on RTW than do conventional German rehabilitation programs; and were designed for patients with strongly limited work ability (e.g., long-term sick leave) and poor self-reported RTW prognosis. A set of randomized controlled trials showed concordantly that increased RTW rates as a result of workrelated multidisciplinary rehabilitation (MR) were measured only in patients with severe restrictions of work ability.11,12 Patients with low or moderate work-related restrictions did not experience any additional benefit from work-related MR.13 Therefore, assessment of the work disability risk and allocation to rehabilitation programs (work-related or conventional MR) according to the identified risk level are of great importance for an effective rehabilitation process.14 In this context, the use of valid screening instruments that measure self-rated work ability could be an appropriate and efficient strategy.15,16 In Germany, a short screening instrument for the identification of severe restrictions of work ability was developed for allocation to work-related or conventional MR (German: Screening-Instrument zur Feststellung des Bedarfs an medizinisch-beruflich orientierter Rehabilitation [SIMBO]). Agreement between the need assessment based on the screening and the need rating of rehabilitation physicians was approximately 80% in one study.17 Another study in patients with several chronic diseases (musculoskeletal, mental, and internal disorders) showed the predictive validity of this screening for RTW.18 In the present article, further results on the predictive validity of the screening for critical RTW follow-up events are reported using a combined meta-sample from several cohorts of rehabilitation patients with CMSDs.
METHODS Setting and Participants This analysis included patient samples from three former prospective cohort studies that assessed similar data and was initially performed to investigate the quality and effects of German MR. One study recruited patients in two rehabilitation centers, whereas the other two studies were singlecenter studies. The sample consisted of patients with different CMSDs who received inpatient rehabilitation on behalf of the German Pension Insurance between 2004 and 2008.17,18 The German Pension Insurance is responsible for rehabilitation service of persons with chronic diseases and a risk for permanent work disability
Am. J. Phys. Med. Rehabil. & Vol. 94, No. 8, August 2015 Copyright © 2015 Wolters Kluwer Health, Inc. All rights reserved.
not caused by occupational accidents. Rehabilitation measures need a format request of the patient in need. An initial need assessment of German Pension Insurance physicians leads then to approval or refusal of a request and is usually done on the basis of the patient’s self-reports and a supporting report of the patient’s general practitioner. After approval of the request, the German Pension Insurance allocated the patients to one of the participating rehabilitation centers. The inpatient rehabilitation followed a multidisciplinary approach. The programs lasted 3Y4 wks (approximately 50Y70 hrs in total) and included exercise, physical therapy, social counseling, patient education (e.g., pain and stress management), as well as passive therapies such as massages and therapeutic baths.19 The multiprofessional team involved physicians, exercise trainers, physical and occupational therapists, psychologists, as well as social workers. Recruitment was usually carried out by the attending physician. After informed consent was obtained, the patients were given a questionnaire at admission. After 6 mos, the researchers sent a postal questionnaire to the participants to collect followup data. In addition, the questionnaire data were linked to the physician-reported data obtained from the standardized rehabilitation discharge form.20 Participants without active employment (e.g., housewives) and those receiving a retirement benefit were excluded. Furthermore, participants with a missing value in the interesting variables were excluded.
The SIMBO The SIMBO score is calculated as the weighted sum of seven single items. The total score ranges from 0 to 100 points (Fig. 1). Higher values indicate an increased risk for permanent work disability and therefore a need for additional work-related interventions according to the recently published guideline for work-related MR.21,22 The single items (unemployment, current sick leave episode, sickness absence of 96 mos during the past year, subjective work disability [0Y10] 9 7 points, expectation of not being able to work at all, motivation for work-related interventions [1Y5] 9 3 points, and age G 46 yrs) are dichotomous criteria that were identified as risk factors of failed RTW as a result of CMSD in former cohort studies and structured reviews23Y26 (summarized in Streibelt and Bethge18). The weights of the single items were empirically derived from prediction models.17 www.ajpmr.com
Outcome Measures and Covariates Critical RTW Events During the 6-mo Follow-up Period The main outcome was failed RTW.16 RTW is, by law, the overall goal of rehabilitation services in Germany. Because sick leave duration is a strong predictor of disability pension, RTW evaluation needs also to consider follow-up sick leave duration. Therefore, RTW was defined as active employment with a maximum of 12 wks of sick leave during the 6-mo follow-up period, whereas failed RTW meant unemployment or active employment but sick leave duration of more than 12 wks during the 6-mo follow-up period. In addition, unemployment vs. employment and employment with sick leave of more than 12 wks vs. employment with sick leave of 12 wks or less were considered as additional outcomes.
Physician-Reported Work Ability Assessment at Discharge The physician’s assessment of the patient’s work ability was obtained from the standardized discharge form. These assessments include recommendations regarding vocational rehabilitation (VR), that is, a qualification measure if a former job can no longer be performed, and the evaluation of daily work time capacity in the current job (96 hrs per day vs. e6 hrs per day). This evaluation is the foundation of further decisions concerning disability pension claims.
Sociodemographics and Covariates Age, sex, and self-rated health (visual analog scale, 0Y100 points) were assessed as covariates. To characterize the sample, socioeconomic status (low, middle, high)27 was also assessed, and the primary treatment diagnosis was obtained from the standardized discharge form.
Analysis Descriptive statistics were used to present the baseline sample characteristics, including the SIMBO score and single SIMBO items as well as the prevalence of the outcome measures. To analyze the predictive characteristics of the SIMBO score for each outcome measure, a receiver operating characteristic curve was constructed, and the area under the curve (AUC) was determined.28 The AUC represents the probability that a randomly selected person with an adverse outcome (e.g., failed RTW) will have a higher SIMBO score than a randomly selected person without this outcome. Values of greater than 0.5 indicate a better prediction than chance. Sensitivity, specificity, and percentage of agreement were calculated using each SIMBO score as a cutoff value to classify persons Predictive Validity of the SIMBO Screening
Copyright © 2015 Wolters Kluwer Health, Inc. All rights reserved.
619
FIGURE 1 Screening for the identification of severe restrictions of work ability (SIMBO) including a calculation guideline.
as either high-risk or low-risk patients. The optimal SIMBO cutoff is then the one that maximizes the Youden index (J), which is defined as sensitivity + specificity j 1.29 In addition, the positive and negative predictive value were calculated for the optimal cutoff. Finally, for each outcome measure, random intercept logistic regression models were calculated to estimate how much higher the odds of an adverse outcome are for the patients with a high SIMBO score, that is, greater than or equal to the optimal cutoff value, compared with the persons with a low SIMBO score, that is, less than the optimal cutoff value. Effect estimates were adjusted for age, sex, and self-rated health. A random intercept was used to account for the clustered data within the study samples. Statistical differences were regarded as significant if the two-sided P value of the test was less than 0.05. All calculations were performed using Stata 10.0.
620
Streibelt and Bethge
RESULTS Sample The four samples comprised a total of 2545 participants. Of these, 1985 (78%) responded to the 6-mo follow-up questionnaire. After the exclusion of participants without active employment and those receiving a retirement benefit (n = 90) as well as participants with a missing SIMBO score or a missing outcome score (n = 140), the remaining sample consisted of 1755 participants. Data from the standardized discharge form were available for only three of the four rehabilitation centers involved (n = 1425). A total of 69.8% of the patients had dorsopathies (International Classification of Diseases, 10th Revision: M40YM55), followed by arthropathies (M00YM25) and soft tissue disorders (M60YM79). The mean age was 47.7 yrs; 34.6% were women. The sample had
Am. J. Phys. Med. Rehabil. & Vol. 94, No. 8, August 2015 Copyright © 2015 Wolters Kluwer Health, Inc. All rights reserved.
a mean (SD) SIMBO score of 24 (23.6) points. The distribution of the SIMBO scores was right skewed. The mass of the patients had low scores, giving further support for the construct validity of the SIMBO scale. The prevalence of the single SIMBO criterion ranged from 8.6% (unemployment) to 65.5% (motivation for work-related MR). The baseline characteristics are reported in Table 1. The physicians stated a need for further VR after MR for 14.7% of the patients. A total of 11.7% were evaluated as able to work for not more than 6 hrs per day. Six months after discharge of MR, 25% of the participants failed RTW; 17.3% were unemployed and 13.6% reported sick leave of more than 12 wks during the 6-mo follow-up period.
SIMBO Score and Failed RTW: The Optimal Cutoff The AUC predicting failed RTW was 0.82 (95% confidence interval [CI], 0.80Y0.84; Table 2 and Fig. 2) and confirmed the predictive validity of the SIMBO score. The AUCs for predicting unemployment (AUC, 0.81; 95% CI, 0.79Y0.83) and long-term sick leave (AUC, 0.82; 95% CI, 0.80Y0.83) were similar. Work
time capacity was also well predicted by the SIMBO score (AUC, 0.89; 95% CI, 0.88Y0.91). The prediction of recommended VR was somewhat worse (AUC, 0.78; 95% CI, 0.76Y0.80) (Fig. 3). The optimal cutoff of the SIMBO score predicting failed RTW was 23 points. Given this cutoff, sensitivity and specificity were 80% and 76%, respectively (Table 3). A total of 77% of the patients were correctly classified. Sensitivity decreased as the cutoff increased and vice versa. Between 15 and 39 points of the SIMBO score, the Youden index was 0.50 or greater, meaning that the mean of sensitivity and specificity was greater than or equal to 75%. Fifty-two percent of those patients tested positive on the SIMBO (Q23 points) failed RTW after rehabilitation (positive predictive value). Ninety-two percent of those patients, who were identified as low-risk patients (G23 points), successfully returned to work after rehabilitation (negative predictive value).
Predicting Critical RTW Events at Follow-up Using the Optimal SIMBO Cutoff The associations of the baseline SIMBO score and follow-up outcomes are presented in Table 4.
TABLE 1 Sample characteristics Characteristic Baseline Age, mean (SD), yrs Sex: female Diagnosis (ICD-10) Arthropathies (M00YM25) Dorsopathies (M40YM54) Soft tissue disorders (M60YM79) Other Social status Low Middle High Self-rated health, mean (SD) SIMBO score, mean (SD) Unemployed Sick listed Sick leave (96 mos) Work disability (97 points) Negative subjective RTW prognosis Motivation for work-related MR Age (G46 yrs) Outcome measures Discharge form Recommended VR Work time capacity in current job (e6 hrs per day) Follow-up Unemployment Sick leave (912 wks) Failed RTW
n
Value
1755 1755
47.7 (8.7) 34.6
168 1225 70 62 1734 302 1182 250 1755 1755 1755
11.0 80.3 4.6 4.1 17.4 68.2 14.4 52.4 (18.0) 24.0 (23.6) 8.6 31.6 12.8 31.2 11.1 65.5 34.8
1417 1412
14.7 11.7
1755 1755 1755
17.3 13.6 25.0
Values are mean (SD) or percentage.
www.ajpmr.com
Predictive Validity of the SIMBO Screening Copyright © 2015 Wolters Kluwer Health, Inc. All rights reserved.
621
TABLE 2 AUC for predicting work-related outcomes using the SIMBO score Outcome Measures Follow-up Failed RTW Unemployment Sick leave (912 wks) Discharge form Work time capacity in current job (e6 hrs per day) Recommended VR
The odds of failed RTW in persons with a baseline SIMBO score of 23 points or greater were 12 times higher than those with a lower score. Moreover, the odds of unemployment and long-term sick leave in this group were 11 times and 10 times higher, respectively. Accordingly, the adjusted prevalence rates of failed RTW events were 0.28Y0.52 for the persons with higher SIMBO scores. Failed RTW events were rare for the persons with low SIMBO scores (0.04Y0.08).
Predicting Physician-Reported Work Ability Assessment at Discharge Using the Optimal SIMBO Cutoff The associations between the baseline SIMBO score and physician-reported work ability assessment at discharge are also reported in Table 4. The odds of decreased capacity in the current job were 20.5 times higher in the patients with SIMBO scores of 23 points or greater. Hardly anyone of the patients with a low SIMBO score was identified as having decreased capacity in their current job. The
n
AUC
95% CI
1755 1755 1755
0.822 0.809 0.815
0.804Y0.840 0.789Y0.827 0.796Y0.833
1412 1417
0.892 0.781
0.875Y0.908 0.758Y0.802
odds of a recommended VR program were also higher for the patients with higher SIMBO scores (odds ratio, 7.2).
Association with Sociodemographic Characteristics Figure 4 shows that the prevalence rate of severe restrictions of work ability is strongly associated with sociodemographic characteristics. Persons with a low social status (i.e., low education or low income) have a higher risk for severe restrictions of work ability. With increasing age, the risk for severe restrictions of work ability decreases steadily until the age of 60 yrs.
DISCUSSION The aim of the present study was to clarify the relevance of the SIMBO screening instrument for identifying patients with a high risk for work disability after a multimodal rehabilitation program. The analyses of this study demonstrate the ability of the SIMBO to predict critical RTW events such as unemployment or long-term sick leave at follow-up
FIGURE 2 Receiver operating characteristic curves for predicting work-related outcomes at follow-up using the baseline SIMBO score.
622
Streibelt and Bethge
Am. J. Phys. Med. Rehabil. & Vol. 94, No. 8, August 2015 Copyright © 2015 Wolters Kluwer Health, Inc. All rights reserved.
FIGURE 3 Receiver operating characteristic curves for predicting physician-reported work ability assessment using the baseline SIMBO score.
as well as physician-reported assessments of work ability. The SIMBO is therefore a useful tool for identifying persons with a high risk for work disability before starting the rehabilitation process. Further studies have shown that administrative or medical decisions usually have low validity when they are based on insufficient information.14,30,31 In such cases, a Bpositive selection[ or Bcreaming[ can be observed, meaning that the physicians or case managers focus on the attributed goal instead of the achieved changes when deciding on access to particular services.32 In this context, screening instruments provide condensed and validated information that is independent of expectations, experiences, and subjective theories. In Germany, the use of screening instruments for access to rehabilitation services is discussed for approximately 10 yrs. Numerous studies in the German rehabilitation system have shown that the quality of decisions concerning the need for rehabilitation33 and access to specialized rehabilitation services34,35 could be improved by using screening
instruments. If the effects of an intervention are proven only for certain patients, proper identification of the target group is needed to maximize outcomes with respect to the reality of budget constraints in the area of rehabilitation. Although there are only few studies that deal with the relevance of screening-based risk profile selection, the authors of a recent review concerning the effectiveness of RTW treatments for sick-listed patients concluded that Bit may be advisable in composing an effective treatment program to take account of the duration of the sick leave before starting treatment or making a return to work prognosis.[36 For persons with a need for rehabilitation, the SIMBO is particularly useful for identifying subgroups with a high risk for permanent work disability as a result of chronic diseases. This offers the opportunity to assign these patients to intensified work-related rehabilitation measures. The prevalence rate of severe restrictions of work ability was 38% in this sample. However, the observed prevalence is dependent on the cutoff
TABLE 3 Sensitivity, specificity, PoA, and Youden index for selected cutoff values of the SIMBO score predicting failed RTW Cutoff Value
Sensitivity, %
Specificity, %
PoA, %
J
87.2 80.6 79.5 73.5 63.9 48.0
54.2 72.7 75.9 79.3 85.3 92.5
62.5 74.6 76.8 77.8 79.9 81.4
0.41 0.53 0.55 0.53 0.49 0.40
10 20 23 30 40 50 PoA, percentage of agreement.
www.ajpmr.com
Predictive Validity of the SIMBO Screening Copyright © 2015 Wolters Kluwer Health, Inc. All rights reserved.
623
TABLE 4 Predicting work-related outcomes: SIMBO score of 23 points or greater vs. SIMBO score of less than 23 points EMMoutcome Outcome Measures Follow-up Failed RTW Unemployment Sick leave (912 wks) Discharge form Work time capacity in current job (e6 hrs per day) Recommended VR
n
OR
95% CI
SIMBO G 23
SIMBO Q 23
1703 1703 1703
11.83 10.59 10.19
8.92Y15.67 7.62Y14.71 6.97Y14.91
0.08 0.05 0.04
0.52 0.37 0.28
1368 1373
20.54 7.19
11.06Y38.14 4.87Y10.62
0.02 0.05
0.27 0.29
All estimates are adjusted for age, sex, and subjective health. Model estimates were used to predict the means of the outcome criteria with balanced factor variables and covariates at their means. EMM, estimated marginal means for the predicted probability of the outcome; OR, odds ratio.
definition. Besides the recommended cutoff value of 23 points, different cutoff values between 15 and 40 points had also acceptable prognostic relevance. This demonstrates the high flexibility of the SIMBO. There is a high association of the prevalence with demographic characteristics. Persons with a low social status have a higher risk for severe restrictions of work ability. Moreover, older persons had a lower risk. The authors assume that this latter association reflects different access patterns to rehabilitation services in Germany. Indeed, the utilization rate of younger patients is rather low, but these patients have very strong limitations. This study has some limitations that need to be considered. First, the merged samples came from different studies, and the data of the original studies
were not collected specifically for this study. Furthermore, the critical RTW events were assessed by self-report. These limitations are balanced by the following strengths: First, the authors used a fairly large sample recruited in several centers. Second, the validity of this study’s findings was improved by a subsidiary analysis with physician-reported outcomes.
CONCLUSIONS Overall, the screening instrument SIMBO predicts work disability after a rehabilitation program in patients with CMSDs. Consequently, the biggest provider of rehabilitation services in Germany, the German Federal Pension Insurance, decided to implement the SIMBO in its routine of rehabilitation
FIGURE 4 Prevalence of SIMBO score of 23 points or greater in several subgroups.
624
Streibelt and Bethge
Am. J. Phys. Med. Rehabil. & Vol. 94, No. 8, August 2015 Copyright © 2015 Wolters Kluwer Health, Inc. All rights reserved.
management of persons with severe restrictions of work ability. Further research should focus on the validity of the instrument in different rehabilitation systems.
REFERENCES 1. Hoefsmit N, Houkes I, Nijhuis FJ: Intervention characteristics that facilitate return to work after sickness absence: A systematic literature review. J Occup Rehabil 2012;22:462Y77 2. Norlund A, Ropponen A, Alexanderson K: Multidisciplinary interventions: Review of studies of return to work after rehabilitation for low back pain. J Rehabil Med 2009;41:115Y21 3. Schaafsma FG, Whelan K, van der Beek AJ, et al: Physical conditioning as part of a return to work strategy to reduce sickness absence for workers with back pain. Cochrane Database Syst Rev 2013;8:CD001822 4. Schaafsma F, Schonstein E, Whelan KM, et al: Physical conditioning programs for improving work outcomes in workers with back pain. Cochrane Database Syst Rev 2010;1:CD001822 5. Saltychev M, Laimi K, El-Metwally A, et al: Effectiveness of multidisciplinary primary prevention in decreasing the risk of work disability in a low-risk population. Scand J Work Environ Health 2012;38:27Y37 6. Saltychev M, Laimi K, Oksanen T, et al: Does perceived work ability improve after a multidisciplinary preventive program in a population with no severe medical problems? The Finnish Public Sector Study. Scand J Work Environ Health 2013;39:57Y65 7. Steenstra IA, Knol DL, Bongers PM, et al: What works best for whom? An exploratory, subgroup analysis in a randomized, controlled trial on the effectiveness of a workplace intervention in low back pain patients on return to work. Spine (Phila Pa 1976) 2009;34:1243Y9 8. Stapelfeldt CM, Christiansen DH, Jensen OK, et al: Subgroup analyses on return to work in sick-listed employees with low back pain in a randomised trial comparing brief and multidisciplinary intervention. BMC Musculoskelet Disord 2011;12:112 9. Schultz IZ, Crook J, Berkowitz J, et al: A prospective study of the effectiveness of early intervention with high-risk back-injured workersVA pilot study. J Occup Rehabil 2008;18:140Y51
13. Streibelt M, Hansmeier T, Mu¨ller-Fahrnow W: Effects of work-related medical rehabilitation in patients with musculoskeletal disorders [in German]. Rehabilitation (Stuttg) 2006;45:161Y71 14. Saltychev M, Laimi K, Oksanen T, et al: Predictive factors of future participation in rehabilitation in the working population: The Finnish public sector study. J Rehabil Med 2011;43:404Y10 15. Bethge M, Radoschewski FM, Gutenbrunner C: The Work Ability Index as a screening tool to identify the need for rehabilitation: Longitudinal findings from the Second German Sociomedical Panel of Employees. J Rehabil Med 2012;44:980Y7 16. Bethge M, Gutenbrunner C, Neuderth S: Work Ability Index predicts application for disability pension after work-related medical rehabilitation for chronic back pain. Arch Phys Med Rehabil 2013;94:2262Y8 17. Streibelt M: Activity and Participation [Effectiveness of Work-Related Interventions in the Medical Rehabilitation on Behalf of the Pension Insurance] [in German]. Lengerich, Germany, Pabst, 2007 18. Streibelt M, Bethge M: Return to Work. Identification of Severe Restrictions of Work Ability in the Field of Medical Rehabilitation [in German]. Lengerich, Germany, Pabst, 2009 19. Deutsche Rentenversicherung: Therapeutical Standards BChronic Back Pain[. Guideline for the Medical Rehabilitation on Behalf of the German Pension Insurance [in German]. Berlin, Germany, Deutsche Rentenversicherung Bund, 2009 20. Deutsche Rentenversicherung Bund: The Physician’s Rehabilitation Discharge Report. Guideline for a Uniform Discharge Report in the Medical Rehabilitation on Behalf of the German Pension Insurance [in German]. Berlin, Germany, Deutsche Rentenversicherung Bund, 2009 21. Streibelt M, Buschmann-Steinhage R: A profile of requirements for the performance of work related medical rehabilitation from the perspective of the statutory pension insurance [in German]. Rehabilitation (Stuttg) 2011;50:160Y7 22. Deutsche Rentenversicherung Bund: Guideline for the Performance of the Work-Related Medical Rehabilitation (WMR) on Behalf of the German Pension Insurance [in German]. Berlin, Germany, Deutsche Rentenversicherung Bund, 2012
10. Lukasczik M, Wolf HD, Gerlich C, et al: Current state of vocationally oriented medical rehabilitationVA German perspective. Disabil Rehabil 2011;33:2646Y55
23. Kuijer W, Groothoff JW, Brouwer S, et al: Prediction of sickness absence in patients with chronic low back pain: A systematic review. J Occup Rehabil 2006;16:439Y67
11. Bethge M, Herbold D, Trowitzsch L, et al: Work status and health-related quality of life following multimodal work hardening: A cluster randomised trial. J Back Musculoskelet Rehabil 2011;24:161Y72
24. Steenstra IA, Verbeek JH, Heymans MW, et al: Prognostic factors for duration of sick leave in patients sick listed with acute low back pain: A systematic review of the literature. Occup Environ Med 2005;62:851Y60
12. Streibelt M, Bethge M: Effects of intensified workrelated multidisciplinary rehabilitation on occupational participation: A randomized-controlled trial in patients with chronic musculoskeletal disorders. Int J Rehabil Res 2014;37:61Y6
25. Mau M, Merkesdal S, Busche T, et al: Prognosis of labour force participation after multidisciplinary outpatient and inpatient rehabilitation for chronic back pain [in German]. Rehabilitation (Stuttg) 2002;41: 160Y6
www.ajpmr.com
Predictive Validity of the SIMBO Screening Copyright © 2015 Wolters Kluwer Health, Inc. All rights reserved.
625
26. Muche R, Ro¨sch M, Flierl S, et al: Development and validation of a prognosis model for predicting work capacity after rehabilitation based on routinely collected parameters [in German]. Rehabilitation (Stuttg) 2000;39:262Y7 27. Winkler J: Measurement of a social status index in the health surveys of the DHP, in Ahrens W, Bellach B, Jo¨ckel KH (eds): Measurement of Sociodemographic Parameters in the Epidemiology [in German]. Berlin, Germany, Robert-Koch-Institut, 1998, pp. 69Y84 28. Hanley JA, McNeil BJ: The meaning and use of the area under a receiver operating characteristic (ROC) curve. Radiology 1982;143:29Y36 29. Youden WJ: Index for rating diagnostic tests. Cancer 1950;3:32Y5 30. Kobelt A, Grosch E, Wasmus A, et al: Is it possible to predict approval of medical rehabilitation by the extent of fatigue and subjective need for rehabilitation? Development, results and acceptance of a short screening [in German]. Rehabilitation (Stuttg) 2007; 46:33Y40 31. Schwarz S, Mangels M, Sohr G, et al: The assignment of orthopaedic rehabilitants to the traditional or the behaviour-medical rehabilitation approachVIs
626
Streibelt and Bethge
it clinically relevant? [in German]. Rehabilitation (Stuttg) 2007;46:323Y32 32. Bell SH, Orr LL: Screening (and creaming?) applicants to job training programs: The AFDC homemaker-home health aide demonstrations. Labour Econ 2002;9:2, 279Y301 33. Meng K, Holderied A, Vogel H: Rehabilitation need in sociomedical assessmentVDevelopment and evaluation of an algorithm of decision-making [in German]. Rehabilitation (Stuttg) 2007;46:41Y9 34. Hu¨ppe A, Parow D, Raspe H: Effectiveness and benefit of a screening procedure to identify patients with diabetes mellitus type 2 in need of rehabilitation: A randomised, controlled trial among members of the Hamburg Mu¨nchener Health Insurance [in German]. Gesundheitswesen 2008;70:590Y9 35. Bethge M, Lo¨ffler S, Schwarz B, et al: Specific workrelated problems: Do they matter in access to workrelated medical rehabilitation? [in German]. Rehabilitation (Stuttg) 2014;53:49Y55 36. Meijer EM, Sluiter JK, Frings-Dresen MH: Evaluation of effective return-to-work treatment programs for sick-listed patients with non-specific musculoskeletal complaints: A systematic review. Int Arch Occup Environ Health 2005;78:523Y32
Am. J. Phys. Med. Rehabil. & Vol. 94, No. 8, August 2015 Copyright © 2015 Wolters Kluwer Health, Inc. All rights reserved.
![[Restrictions of work ability].](/assets/img/hold.png)
