JONA Volume 44, Number 5, pp 263-269 Copyright B 2014 Wolters Kluwer Health | Lippincott Williams & Wilkins
THE JOURNAL OF NURSING ADMINISTRATION
The Baptist Health High Risk Falls Assessment A Methodological Study Donna Corley, PhD, RN, CNE Dorothy Brockopp, PhD, RN Denise McCowan, MSN, RN Sharon Merritt, BSN, RN Teresa Cobb, ADN, RN
Brenda Johnson, MSN, RN Cheryl Stout, MSN, RN, CENP Krista Moe, PhD Brittany Hall, AASB
OBJECTIVE: The purpose of this study was to evaluate the psychometric properties of the Baptist Health High Risk Falls Assessment (BHHRFA). BACKGROUND: The cost of falls and the potential for increased patient morbidity and mortality necessitate the development of a psychometrically sound assessment. METHODS: Based on pilot data, a multisite study was conducted. Assessments were collected on 21 medicalsurgical units, 1 rehabilitation unit, 1 psychiatric unit, and 3 critical care units (N = 241599). Sensitivity, specificity, and diagnostic odds ratios (DORs) were calculated. Clinical usefulness in terms of assessment performance time was addressed. RESULTS: Psychometric properties were within a desirable range for each site, critical care patients, and all participants (sensitivity, 0.64-1.00; specificity,
0.50-0.70; and DOR, 4.73-7.0). Although sensitivity at 1 site was high (0.84), specificity was low (0.37). CONCLUSIONS: Results suggest that the BHHRFA is clinically useful for adult inpatients across acute care settings.
Author Affiliations: Evidence-Based Practice Consultant (Dr Corley), Evidence-Based Practice Coordinator (Dr Brockopp), Nursing Director (Ms McCowan), Magnet/Nursing Special Projects Coordinator (Ms Merritt), Evidence-Based Practice Consultant (Dr Moe), Patient Care Technician (Ms Hall), Baptist Health Lexington, Kentucky; Executive Nursing Director (Ms Cobb), Baptist Health Corbin, Kentucky; Infection Control Nurse and Patient Safety Officer (Ms Johnson), Baptist Health La Grange, Kentucky; Nursing Director (Ms Stout) Baptist Health Louisville, Kentucky. The editor-in-chief of Journal of Nursing Administration is the chief operating officer/CNO of Baptist Health Lexington. John Welton, PhD, RN, professor and senior scientist, University of Colorado, College of Nursing, served as the editor of this submission. The authors declare no conflicts of interest. Correspondence: Dr Brockopp, Baptist Health Lexington, 1740 Nicholasville Rd, Lexington, KY 40503 ([email protected]). DOI: 10.1097/NNA.0000000000000065
JONA Vol. 44, No. 5 May 2014
Accidental falls are the most common patient safety incidents in acute care settings. Falls are costly and can result in mild to serious injuries, increased length of stay, and discharge to a long-term care facility rather than home.1 By 2020, the annual indirect and direct cost of falls in the United States is expected to reach $54.9 billion.2 Fall-related deaths are increasing despite a Healthy People 2010 goal to decrease the number of fall-related deaths among individuals 65 years or older.3 Great efforts have been made to diminish falls in the hospital setting; however, national fall rates have not decreased. The fact that inpatient falls are not decreasing may be related to (1) an increase in the aging population (individuals 965 years old are more likely to fall), (2) the presence of more complex and severely ill patients (result of improved disease management), (3) a lack of comprehensive programs shown to diminish fall rates,4 and (4) a lack of a high-risk falls assessment that is clinically useful and has consistent strong sensitivity, specificity, and diagnostic odds ratios (DORs) across settings.5 Although numerous interventions are suggested in the literature to prevent falls, an instrument with strong diagnostic predictability that is both clinically useful in terms of expenditure of time and energy and predicts falls consistently across clinical settings is not
263
Copyright © 2014 Wolters Kluwer Health | Lippincott Williams & Wilkins. Unauthorized reproduction of this article is prohibited.
available. An examination of 3 frequently used highrisk falls assessments, the Morse Falls Scale (MFS),7 the St. Thomas Risk Assessment Tool in Falling Elderly Inpatients (STRATIFY),8 and the Hendrich II Fall Risk Model (HFRM),9 revealed a number of problems. Concerns regarding these instruments include the following: setting has been used during the development process, assessments have low or inconsistent levels of specificity (the test’s ability to identify negative results) and sensitivity (the test’s ability to identify positive results),1 and/or there is ambiguity of factors and reproducibility across settings.6 Although not consistently reported in the literature, the time taken to assess for risk of falling is another important factor given the high acuity levels of inpatients and associated demands on nurses’ time.
Review of the Literature Findings related to the predictive validity and the psychometric properties of commonly used high-risk falls assessments vary across studies. Aranda-Gallardo et al10 conducted an in-depth analysis of existing literature describing the psychometric properties of the MFS,7 STRATIFY,8 and HFRM II scales. Fourteen studies were selected based on investigators’ focus on development of diagnostic validity, development of psychometric properties, and evaluation of effectiveness of fall risk assessment. Their synthesis included sensitivity and specificity calculations, meta-analyses of DORs and likelihood ratio coefficients, summary receiver operating characteristic curves, and Cochrane Q statistic. Upon examination of the DOR and sensitivity/ specificity of these 3 instruments, the STRATIFY was found to be the most effective approach to assessing the risk of falling among hospitalized adult inpatients. Another review focused on the same 3 instruments reported the sensitivity and specificity for each instrument as follows: HFRM II, 70%/61.5%; MFS, 88.3%/48.3%; and STRATIFY, 55%/75.3%.6 Based on these criteria only, without the DOR, the HFRM II would be most effective as a predictor of falls. However, another review reports sensitivity/specificity as 77%/72% for the HFRM, 79%/82% for the MFS, and 93%/68% for the STRATIFY.11 Inconsistencies in study findings question the applicability of instruments across settings. Differences may be attributed to hospital size, instrument used, sample size, or patient population. There are also inconsistencies in time required to complete assessments. Time reported in the literature varies from ‘‘approximately’’ 1 minute (HFRM II)11 to 3.85 minutes (STRATIFY).12 Additional problems related to research to date include use of multiple cutoff points, poor sensitivity and/or specificity, testing of instruments in inappropriate settings, and discrepancies
264
between original design of instrument and later modifications.10 Although the literature reports inconsistencies in the psychometric properties of available instruments assessing patients at high risk of falling, there are common elements across assessments. These elements are history of falls8,9,13; some form of confusion such as agitation, disorientation, or mental state8,9,13; toileting issues8,14; and mobility problems.8,9,13
About the Study Before developing the Baptist Health High Risk Falls Assessment (BHHRFA), 2 pilot projects related to patient falls were conducted at Baptist Health Lexington (BHLex) in 2010 and 2011. In general, the literature describes reasons for patients’ falling from the healthcare providers’ view.15 The 1st project was designed to better understand why patients fall from the patient’s perspective. A total of 118 inpatients at 2 community hospitals were interviewed within 24 hours after falling and were asked why they fell and whether they asked for help. If they did not ask for help, they were asked why. Responses to open-ended questions included the following: patients did not want to ‘‘bother nurses,’’ they felt an ‘‘urgent’’ need to get out of bed (eg, toileting, picking something up, or getting a drink), and they did not follow healthcare providers’ advice ‘‘not to get up without assistance.’’ Most patients in the 1st pilot project were older than 65 years, had a history of falling, and were documented in their chart as alert. In addition, patients described effects of their medication as possibly contributing to their fall. The 2nd pilot project focused on the development of a high-risk falls assessment that would predict falls at BHLex. Items were derived from the findings of the 1st pilot project and the literature. Table 1 reports items used in the 1st draft of the instrument. This draft was given to 609 patients (upon admission) over a 6-month period and data were analyzed regarding items associated with patient falls. Results of # 2 tests of independence indicated that 6 of the 9 risk factors were associated with patient falls (Table 1). The BHHRFA contains those 6 factors with modest changes in wording and the addition of the item ‘‘nurses’ clinical judgment’’ (Figure 1). This item was added given research that supports the notion that nurses, through their clinical experience, are able to predict those patients who will fall.15,16 Although nurses gave a correct clinical judgment in 35.3% of 79 cases in 1 study, a significant association was not found between nurses’ judgment and patient fall status.15 The purpose of the current study was to examine the psychometric properties of the BHHRFA, an instrument designed to predict those inpatients who are
JONA Vol. 44, No. 5 May 2014
Copyright © 2014 Wolters Kluwer Health | Lippincott Williams & Wilkins. Unauthorized reproduction of this article is prohibited.
Table 1. # 2 Tests of Independence: Pilot Study on Patient Falls and Risk Factors Risk Factor
# 2 (1, n = 609)
P
8
History of falls Fall in past 6 months Q60 years old Impaired cognition Incontinence Frequent toileting Mobility issues Sensory problems Medications
51.89 (1,1305) 23.95 (1,1307) 2.58 (1,1337) 78.91 (1,1336) 0.13 (1,1337) 11.89 (1,1337) 16.64 (1,1337) 6.97 (1,1335) 0.001 (1,1336)
.0005 .0005 .275 .0005 .71 .001 .0005 .008 .977
0.20 0.14 0.04 0.24 0.01 0.09 0.11 0.07 0.01
Of the 609 patients, 21 fell.
likely to fall. The design of the BHHRFA was based on a comprehensive review of the literature and results of 2 pilot projects. Projected outcomes of the project were to produce an instrument to assess patients at high risk for falling that had desirable specificity, sensitivity, and DOR across settings and would take approximately 1 minute to complete.
Methods Based on data retrieved from the 2 pilot projects, the 7-item BHHRFA assessment was designed (Figure 1). Selection of items was based on data retrieved from the 2 pilot projects conducted before this study as well as a comprehensive review of the literature. The item assessing nurse’s judgment regarding fall risk was added based on a study that suggests nursing judgment may make a valuable contribution to assessment of risk.17 A total score of 13 or above was designated as high risk for falling. The cutoff score as well as points given to each item were determined based on pilot data, expert opinion and the literature. After approval of the organization’s institutional review board was obtained, a multisite study was conducted at BHLex (urban, 383 beds) and 3 additional hospitals within the Baptist Healthcare System. Administration at each setting, Hospital A (rural, 120 beds), Hospital B (rural, 241 beds), and Hospital C (urban, 519 beds), agreed to participate in the project. Data collection was initiated in June 2012 and completed December 2013. Using the BHHRFA, nurses at each institution collected data once per shift on each of their patients. Data (an assessment of each patient’s falls risk) were collected from 7 adult medical-surgical units (BHLex), 3 intensive care units (BHLex), 1 rehabilitation unit (hospital C), 1 psychiatric unit (hospital C), 4 medical-surgical units (hospital C), 2 medical-surgical units (hospital A), and 3 medical-surgical units (hospital B) for a total of 18 units. A test of the instrument in critical care was conducted at the lead hospital, given
JONA Vol. 44, No. 5 May 2014
an interest in incorporating the Confusion Assessment Method18 score into the BHHRFA. The BHHRFAs were collected by nurse managers, transported to the nursing research office, and entered into a statistics program by research assistants. At the other sites, assessments were scanned into electronic files for access, data entry, and analysis at BHLex. Training sessions were held to assist nurses to complete the BHHRFA accurately. During these sessions, registered nurses practiced completing assessments based on case studies. A total of 241599 assessments were collected (BHLex, n = 186875; hospital A, n = 6004; hospital B, n = 15270; and hospital C, n = 33000). The number of falls at each site were as follows: BHLex, n = 216; hospital A, n = 4; hospital B, n = 8; and hospital C, n = 70. Total hospital falls at all sites equaled 298 (0.12%). Data collected on patient assessments from each site, patients in critical care, and aggregate data from patients at all hospitals were analyzed using SPSS version 21 (SPSS IBM, New York, New York).19 Sensitivity, specificity, and the DOR were calculated. Logistic regression was used to compare the predictive validity of each of the BHHRFA items. A total of 134 nurses (medical-surgical, n = 114; critical care nurses, n = 20) were timed as they used the BHHRFA and an average time was calculated.
Results Age and gender of participants as well as the shifts (am and pm) when patients were assessed are presented in Table 2. Calculations for sensitivity, specificity, and DOR are presented in Table 3. Range of sensitivity for 4 hospitals was 0.64 to 1.00, specificity for 3 hospitals was 0.50 to 0.75, and DOR for all sites was 4.73 to 7. Although sensitivity at 1 site was high (0.84), specificity was low (0.37). The logistic regression analysis is shown in Table 4. For the purpose of this analysis, the medication list on the BHHRFA was collapsed into ‘‘medication’’ and ‘‘no medication.’’ The model
265
Copyright © 2014 Wolters Kluwer Health | Lippincott Williams & Wilkins. Unauthorized reproduction of this article is prohibited.
Figure 1. The BHHRFA. Reprinted with permission.
266
JONA Vol. 44, No. 5 May 2014
Copyright © 2014 Wolters Kluwer Health | Lippincott Williams & Wilkins. Unauthorized reproduction of this article is prohibited.
Table 2. Age, Gender, and Nursing Shift When Data Were Collected for All Hospital Sites Age, Mean (SD) Baptist Health Hospital BHLex BHLexVICU Hospital A Hospital B Hospital C All hospitals (medical-surgical)
65.50 65.31 65.07 65.90 68.06 65.91
Gender (Female), % (n)
(16.06) (13.73) (17.03) (16.72) (15.52) (16.08)
56.8 (91,019) 45.4 (12,071) 53.9 (3,210) 60.5 (9,515) 57.6 (19,019) 57.1 (122,763)
that includes has fallen in the last 6 months, age, mental status, elimination, mobility, medications, and nurse’s clinical judgment was statistically significant (# 2[N = 214,998] = 329.37, P G .001). Together, these variables are able to distinguish between patients who would and would not fall. Five of the independent variables appeared to make a unique statistically significant contribution to the model (falls in the past 6 months, age, mental status, elimination, and nurses’ judgment). The strongest predictor of falls was ‘‘has fallen in the last 6 months’’ (odds ratio, 2.98). This value indicates that patients who have fallen in the past 6 months are approximately 3 times more likely to fall than those who have not fallen in the past 6 months, controlling for other factors in the model. Time taken for nurses (n = 134) to complete the BHHRFA ranged from 0.33 seconds to 3.00 minutes, with a mean (SD) time of 1.19 (0.56) minutes. Nurses taking more than 1 minute were new graduates. Critical care nurses (n = 20) took less time to complete the assessment (mean [SD], 0.63 [0.15] minutes) than medical-surgical nurses (n = 114) (mean [SD], 1.29 [0.54] minutes). Differences in volume and acuity of patients in medical-surgical and critical care areas may have contributed to this difference in time.
Discussion Age, gender, and the shift when assessments were administered were approximately the same across study
Shift am, % (n) 53.2 (85,284) 50.0 (13,302) 49.8 (2,987) 51.0 (8,015) 51.0 (16,819) 52.6 (113,105)
Shift pm, % (n) 46.8 (74,974) 50.0 (13,298) 50.2 (3,017) 49.0 (7,705) 49.0 (16,181) 47.4 (101,877)
sites. Results of data analyses show that the BHHRFA has strong sensitivity and DORs across 4 settings. Specificity was strong across 3 of the settings and low at 1 hospital (hospital C). A low specificity value at 1 site may be related to nurse behaviors in administering the BHHRFA. For example, an increasing elderly presence in acute care settings who have complex problems and therefore appear to be at higher risk for falling may lead nurses to rate patients at a higher risk for falls than necessary. They may have a bias regarding a particular patient population as likely to fall, and therefore, their ability to predict those who will not fall is low. In summary, regression analyses show that the model used in the development of this instrument is effective in terms of identifying those individuals who will fall. In relation to clinical usefulness, conducting this assessment takes nurses approximately 1 minute. Anecdotal data suggest that variation in time for nurses to complete the BHHRFA may be due to a nurse’s level of experience. For example, a more experienced nurse may take less time to complete the assessment than a new graduate. Limitations Although 4 sites (2 urban, 2 rural) were used to test the BHHRFA, all institutions were within 1 state and are part of the same corporation. Hospitals in different geographic locations within other corporate structures could find the BHHRFA more or less useful. In addition, this assessment was limited to adults; patients
Table 3. Sensitivity, Specificity, and DOR for the BHHRFA for All Hospital Sites Study Site
Hospital Sample Size (n)
BHLex BHLexVICU Hospital A Hospital B Hospital C All hospitals (medical-surgical)
160,274 26,601 6,004 15,720 33,000 214,998
Sensitivity (95% CI) 0.64 0.73 1.00 0.88 0.84 0.70
(0.58-0.71) (0.43-0.90) (0.51-1.00) (0.53-0.98) (0.74-0.91) (0.65-0.75)
Specificity (95% CI) 0.75 0.67 0.58 0.50 0.37 0.66
(0.74-0.75) (0.66-0.67) (0.57-0.60) (0.49-0.51) (0.37-0.38) (0.66-0.67)
DOR (95% CI) 5.29 (3.97-7.04) 5.34 (1.42-20.13) N/Aa 7.0 (0.86-56.91) 3.19 (1.68-6.10) 4.73 (3.70-6.10)
Abbreviation: CI, confidence interval. a DOR not available because of a sensitivity value of 1.
JONA Vol. 44, No. 5 May 2014
267
Copyright © 2014 Wolters Kluwer Health | Lippincott Williams & Wilkins. Unauthorized reproduction of this article is prohibited.
Table 4. Logistic Regression Predicting Likelihood of Falls in an Acute Care Setting Among Medical-Surgical Patients at All Sites (N = 214,998) Predictor Fall in the past 6 months Age group Mental status Elimination Mobility issue Medication Nurse’s judgment (0-10)
B a
1.09 j0.60a 0.77a 0.55a 0.18 0.36 0.19a
SE
Wald
df
Odds Ratio
95% CI
0.13 0.13 0.14 0.13 0.18 0.16 0.03
72.98 21.96 31.65 16.95 1.01 5.23 54.20
1 1 1 1 1 1 1
2.98 .55 2.16 1.73 1.20 1.43 1.20
2.32-3.83 0.43-0.71 1.65-2.82 1.33-2.25 0.84-1.71 1.05-1.93 1.15-1.26
Abbreviation: CI, confidence interval. a P G .001.
younger than 18 years were not included in the study. Only aggregate data were recorded and analyzed. Patients were assessed twice a day during their hospital admission and no attempt was made to differentiate multiple assessments or falls of individual patients.
attend to assessment scores and intervene with those patients deemed likely to fall. Numerous interventions designed to prevent falls are suggested in the literature and need to be tailored to the specific institution and patient population.
Conclusions and Implications for Practice
Recommendations
The development of the BHHRFA spanned a 5-year period. The 2 pilot projects as well as a continuing and comprehensive review of the literature formed the basis for the instrument. The collection of data across multiple sites and the size of the sample are strengths of this project. Although use of an assessment to predict falls is a crucial component of a falls prevention program, prediction alone is not sufficient to diminish the incidence of falls. Healthcare professionals must
Based on the outcomes of this study, further testing of the BHHRFA in different geographic locations and corporate structures is recommended. Positive outcomes regarding sensitivity, specificity, ease of use, and DOR analyses suggest use of this assessment with inpatients in acute care settings. Based on logistic regression analyses, a shorter version of the assessment might be possible. Continued testing of the BHHRFA is recommended.
References 1. Milisen K, Staelens N, Schwendimann R, et al. Fall prediction in inpatients by bedside nurses using the St. Thomas’s Rick Assessment Tool in falling elderly inpatients (STRATIFY) instrument: a multicenter study. J Am Geriatr Soc. 2007;55:725-733. 2. Englander F, Hodson TJ, Terregrossa RA. Economic dimensions of slip and fall injuries. J Forensic Sci. 1996;41(5):733-746. 3. Stevens JA, Mack KA, Paulozzi LJ, Ballesteros MF. Selfreported falls and fall-related injuries among persons aged Q65 yearsVUnited States, 2006. J Saf Res. 2008;39:345-349. 4. Rutledge DN, Donaldson NE, Pravikoff DS. Update 2003: fall risk assessment and prevention in hospitalized patients. Online J Crit Innov. 2003;6(5):1-55. 5. McAlister S. APR DRG weights and the relationship to patient falls. Nurs Econ. 2009;27(2):119-123. 6. Kim EA, Mordiffi SZ, Bee WH, Devi K, Evans D. Evaluation of three fall-risk assessment tools in an acute care setting. J Adv Nurs. 2007;60(4):427-435. doi:10.1111/j.1365-2648.2007.04419.x 7. Morse JM, Morse RM, Tylko SJ. Development of a scale to identify the fall-prone patient. Can J Aging. 1989;8(4):366-377. doi:10.1017/S0714980800008576 8. Oliver D, Britton M, Seed P, et al. Development and evaluation of an evidence based risk assessment tool (STRATIFY) to
268
9.
10.
11.
12. 13.
14.
15.
predict which elderly inpatients will fall: case-control and cohort studies. Br Med J. 1997;315:1049-1053. Hendrich A, Bender P, Nyhuis A. Validation of the Hendrich II Risk Model: a large concurrent case/control study of hospitalized patients. Appl Nurs Res. 2003;16(1):9-21. Aranda-Gallardo M, Morales-Asencio JM, Canca-Sanchez JC, et al. Instrument for assessing the risk of falls in acute hospitalized patients: a systematic review and meta-analysis. BMC Health Serv Res. 2013;13:122. Ivziku D, Matarese M, Pedone C. Predictive validity of the Hendrich fall risk model II in an acute geriatric unit. Int J Nurs Stud. 2011;48:468-474. Morse J. Computerized evaluation of a scale to identify the fallprone patient. Can J Public Health. 1986;77:21-25. Vassallo M, Stockdale R, Sharma J, et al. A comparative study of the use of four fall risk assessment tools on acute medical wards. J Am Geriatr Soc. 2005;53:1034-1038. Rauch K, Balascio J, Gilbert P. Excellence in action: developing and implementing a fall prevention program. J Healthc Qual. 2009;31(1):36-42. Meyer G, Kopke S, Haastert B, et al. Comparison of a fall risk assessment tool with nurses’ clinical judgment alone: a
JONA Vol. 44, No. 5 May 2014
Copyright © 2014 Wolters Kluwer Health | Lippincott Williams & Wilkins. Unauthorized reproduction of this article is prohibited.
cluster-randomized controlled trial. Age Ageing. 2009;38(4): 417-423. 16. Myers H, Nikoletti S. Fall risk assessment: a prospective investigation of nurses’ clinical judgment and risk assessment tools in predicting patient falls. Int J Nurs Pract. 2003;9(3): 158-165. 17. Choi YS, Lawler E, Boenecke CA, Ponatoski ER, Zimring CM. Developing a multi-systemic fall prevention model, incorporat-
JONA Vol. 44, No. 5 May 2014
ing the physical environment, the care process and technology: a systematic review. J Adv Nurs. 2011;67(12):2501-2524. doi:10 .1111/j.1365-2648.2011.05672.x 18. Inouye SK, van Dyck CH, Alessi CA, et al. Clarifying confusion: the confusion assessment method a new method of detection of delirium. Ann Intern Med. 1990;113(12):941-948. 19. IBM Corp. IBM SPSS Statistics for Windows, Version 21.0. Armonk, NY: IBM Corp; 2012.
269
Copyright © 2014 Wolters Kluwer Health | Lippincott Williams & Wilkins. Unauthorized reproduction of this article is prohibited.
THE JOURNAL OF NURSING ADMINISTRATION
The Baptist Health High Risk Falls Assessment A Methodological Study Donna Corley, PhD, RN, CNE Dorothy Brockopp, PhD, RN Denise McCowan, MSN, RN Sharon Merritt, BSN, RN Teresa Cobb, ADN, RN
Brenda Johnson, MSN, RN Cheryl Stout, MSN, RN, CENP Krista Moe, PhD Brittany Hall, AASB
OBJECTIVE: The purpose of this study was to evaluate the psychometric properties of the Baptist Health High Risk Falls Assessment (BHHRFA). BACKGROUND: The cost of falls and the potential for increased patient morbidity and mortality necessitate the development of a psychometrically sound assessment. METHODS: Based on pilot data, a multisite study was conducted. Assessments were collected on 21 medicalsurgical units, 1 rehabilitation unit, 1 psychiatric unit, and 3 critical care units (N = 241599). Sensitivity, specificity, and diagnostic odds ratios (DORs) were calculated. Clinical usefulness in terms of assessment performance time was addressed. RESULTS: Psychometric properties were within a desirable range for each site, critical care patients, and all participants (sensitivity, 0.64-1.00; specificity,
0.50-0.70; and DOR, 4.73-7.0). Although sensitivity at 1 site was high (0.84), specificity was low (0.37). CONCLUSIONS: Results suggest that the BHHRFA is clinically useful for adult inpatients across acute care settings.
Author Affiliations: Evidence-Based Practice Consultant (Dr Corley), Evidence-Based Practice Coordinator (Dr Brockopp), Nursing Director (Ms McCowan), Magnet/Nursing Special Projects Coordinator (Ms Merritt), Evidence-Based Practice Consultant (Dr Moe), Patient Care Technician (Ms Hall), Baptist Health Lexington, Kentucky; Executive Nursing Director (Ms Cobb), Baptist Health Corbin, Kentucky; Infection Control Nurse and Patient Safety Officer (Ms Johnson), Baptist Health La Grange, Kentucky; Nursing Director (Ms Stout) Baptist Health Louisville, Kentucky. The editor-in-chief of Journal of Nursing Administration is the chief operating officer/CNO of Baptist Health Lexington. John Welton, PhD, RN, professor and senior scientist, University of Colorado, College of Nursing, served as the editor of this submission. The authors declare no conflicts of interest. Correspondence: Dr Brockopp, Baptist Health Lexington, 1740 Nicholasville Rd, Lexington, KY 40503 ([email protected]). DOI: 10.1097/NNA.0000000000000065
JONA Vol. 44, No. 5 May 2014
Accidental falls are the most common patient safety incidents in acute care settings. Falls are costly and can result in mild to serious injuries, increased length of stay, and discharge to a long-term care facility rather than home.1 By 2020, the annual indirect and direct cost of falls in the United States is expected to reach $54.9 billion.2 Fall-related deaths are increasing despite a Healthy People 2010 goal to decrease the number of fall-related deaths among individuals 65 years or older.3 Great efforts have been made to diminish falls in the hospital setting; however, national fall rates have not decreased. The fact that inpatient falls are not decreasing may be related to (1) an increase in the aging population (individuals 965 years old are more likely to fall), (2) the presence of more complex and severely ill patients (result of improved disease management), (3) a lack of comprehensive programs shown to diminish fall rates,4 and (4) a lack of a high-risk falls assessment that is clinically useful and has consistent strong sensitivity, specificity, and diagnostic odds ratios (DORs) across settings.5 Although numerous interventions are suggested in the literature to prevent falls, an instrument with strong diagnostic predictability that is both clinically useful in terms of expenditure of time and energy and predicts falls consistently across clinical settings is not
263
Copyright © 2014 Wolters Kluwer Health | Lippincott Williams & Wilkins. Unauthorized reproduction of this article is prohibited.
available. An examination of 3 frequently used highrisk falls assessments, the Morse Falls Scale (MFS),7 the St. Thomas Risk Assessment Tool in Falling Elderly Inpatients (STRATIFY),8 and the Hendrich II Fall Risk Model (HFRM),9 revealed a number of problems. Concerns regarding these instruments include the following: setting has been used during the development process, assessments have low or inconsistent levels of specificity (the test’s ability to identify negative results) and sensitivity (the test’s ability to identify positive results),1 and/or there is ambiguity of factors and reproducibility across settings.6 Although not consistently reported in the literature, the time taken to assess for risk of falling is another important factor given the high acuity levels of inpatients and associated demands on nurses’ time.
Review of the Literature Findings related to the predictive validity and the psychometric properties of commonly used high-risk falls assessments vary across studies. Aranda-Gallardo et al10 conducted an in-depth analysis of existing literature describing the psychometric properties of the MFS,7 STRATIFY,8 and HFRM II scales. Fourteen studies were selected based on investigators’ focus on development of diagnostic validity, development of psychometric properties, and evaluation of effectiveness of fall risk assessment. Their synthesis included sensitivity and specificity calculations, meta-analyses of DORs and likelihood ratio coefficients, summary receiver operating characteristic curves, and Cochrane Q statistic. Upon examination of the DOR and sensitivity/ specificity of these 3 instruments, the STRATIFY was found to be the most effective approach to assessing the risk of falling among hospitalized adult inpatients. Another review focused on the same 3 instruments reported the sensitivity and specificity for each instrument as follows: HFRM II, 70%/61.5%; MFS, 88.3%/48.3%; and STRATIFY, 55%/75.3%.6 Based on these criteria only, without the DOR, the HFRM II would be most effective as a predictor of falls. However, another review reports sensitivity/specificity as 77%/72% for the HFRM, 79%/82% for the MFS, and 93%/68% for the STRATIFY.11 Inconsistencies in study findings question the applicability of instruments across settings. Differences may be attributed to hospital size, instrument used, sample size, or patient population. There are also inconsistencies in time required to complete assessments. Time reported in the literature varies from ‘‘approximately’’ 1 minute (HFRM II)11 to 3.85 minutes (STRATIFY).12 Additional problems related to research to date include use of multiple cutoff points, poor sensitivity and/or specificity, testing of instruments in inappropriate settings, and discrepancies
264
between original design of instrument and later modifications.10 Although the literature reports inconsistencies in the psychometric properties of available instruments assessing patients at high risk of falling, there are common elements across assessments. These elements are history of falls8,9,13; some form of confusion such as agitation, disorientation, or mental state8,9,13; toileting issues8,14; and mobility problems.8,9,13
About the Study Before developing the Baptist Health High Risk Falls Assessment (BHHRFA), 2 pilot projects related to patient falls were conducted at Baptist Health Lexington (BHLex) in 2010 and 2011. In general, the literature describes reasons for patients’ falling from the healthcare providers’ view.15 The 1st project was designed to better understand why patients fall from the patient’s perspective. A total of 118 inpatients at 2 community hospitals were interviewed within 24 hours after falling and were asked why they fell and whether they asked for help. If they did not ask for help, they were asked why. Responses to open-ended questions included the following: patients did not want to ‘‘bother nurses,’’ they felt an ‘‘urgent’’ need to get out of bed (eg, toileting, picking something up, or getting a drink), and they did not follow healthcare providers’ advice ‘‘not to get up without assistance.’’ Most patients in the 1st pilot project were older than 65 years, had a history of falling, and were documented in their chart as alert. In addition, patients described effects of their medication as possibly contributing to their fall. The 2nd pilot project focused on the development of a high-risk falls assessment that would predict falls at BHLex. Items were derived from the findings of the 1st pilot project and the literature. Table 1 reports items used in the 1st draft of the instrument. This draft was given to 609 patients (upon admission) over a 6-month period and data were analyzed regarding items associated with patient falls. Results of # 2 tests of independence indicated that 6 of the 9 risk factors were associated with patient falls (Table 1). The BHHRFA contains those 6 factors with modest changes in wording and the addition of the item ‘‘nurses’ clinical judgment’’ (Figure 1). This item was added given research that supports the notion that nurses, through their clinical experience, are able to predict those patients who will fall.15,16 Although nurses gave a correct clinical judgment in 35.3% of 79 cases in 1 study, a significant association was not found between nurses’ judgment and patient fall status.15 The purpose of the current study was to examine the psychometric properties of the BHHRFA, an instrument designed to predict those inpatients who are
JONA Vol. 44, No. 5 May 2014
Copyright © 2014 Wolters Kluwer Health | Lippincott Williams & Wilkins. Unauthorized reproduction of this article is prohibited.
Table 1. # 2 Tests of Independence: Pilot Study on Patient Falls and Risk Factors Risk Factor
# 2 (1, n = 609)
P
8
History of falls Fall in past 6 months Q60 years old Impaired cognition Incontinence Frequent toileting Mobility issues Sensory problems Medications
51.89 (1,1305) 23.95 (1,1307) 2.58 (1,1337) 78.91 (1,1336) 0.13 (1,1337) 11.89 (1,1337) 16.64 (1,1337) 6.97 (1,1335) 0.001 (1,1336)
.0005 .0005 .275 .0005 .71 .001 .0005 .008 .977
0.20 0.14 0.04 0.24 0.01 0.09 0.11 0.07 0.01
Of the 609 patients, 21 fell.
likely to fall. The design of the BHHRFA was based on a comprehensive review of the literature and results of 2 pilot projects. Projected outcomes of the project were to produce an instrument to assess patients at high risk for falling that had desirable specificity, sensitivity, and DOR across settings and would take approximately 1 minute to complete.
Methods Based on data retrieved from the 2 pilot projects, the 7-item BHHRFA assessment was designed (Figure 1). Selection of items was based on data retrieved from the 2 pilot projects conducted before this study as well as a comprehensive review of the literature. The item assessing nurse’s judgment regarding fall risk was added based on a study that suggests nursing judgment may make a valuable contribution to assessment of risk.17 A total score of 13 or above was designated as high risk for falling. The cutoff score as well as points given to each item were determined based on pilot data, expert opinion and the literature. After approval of the organization’s institutional review board was obtained, a multisite study was conducted at BHLex (urban, 383 beds) and 3 additional hospitals within the Baptist Healthcare System. Administration at each setting, Hospital A (rural, 120 beds), Hospital B (rural, 241 beds), and Hospital C (urban, 519 beds), agreed to participate in the project. Data collection was initiated in June 2012 and completed December 2013. Using the BHHRFA, nurses at each institution collected data once per shift on each of their patients. Data (an assessment of each patient’s falls risk) were collected from 7 adult medical-surgical units (BHLex), 3 intensive care units (BHLex), 1 rehabilitation unit (hospital C), 1 psychiatric unit (hospital C), 4 medical-surgical units (hospital C), 2 medical-surgical units (hospital A), and 3 medical-surgical units (hospital B) for a total of 18 units. A test of the instrument in critical care was conducted at the lead hospital, given
JONA Vol. 44, No. 5 May 2014
an interest in incorporating the Confusion Assessment Method18 score into the BHHRFA. The BHHRFAs were collected by nurse managers, transported to the nursing research office, and entered into a statistics program by research assistants. At the other sites, assessments were scanned into electronic files for access, data entry, and analysis at BHLex. Training sessions were held to assist nurses to complete the BHHRFA accurately. During these sessions, registered nurses practiced completing assessments based on case studies. A total of 241599 assessments were collected (BHLex, n = 186875; hospital A, n = 6004; hospital B, n = 15270; and hospital C, n = 33000). The number of falls at each site were as follows: BHLex, n = 216; hospital A, n = 4; hospital B, n = 8; and hospital C, n = 70. Total hospital falls at all sites equaled 298 (0.12%). Data collected on patient assessments from each site, patients in critical care, and aggregate data from patients at all hospitals were analyzed using SPSS version 21 (SPSS IBM, New York, New York).19 Sensitivity, specificity, and the DOR were calculated. Logistic regression was used to compare the predictive validity of each of the BHHRFA items. A total of 134 nurses (medical-surgical, n = 114; critical care nurses, n = 20) were timed as they used the BHHRFA and an average time was calculated.
Results Age and gender of participants as well as the shifts (am and pm) when patients were assessed are presented in Table 2. Calculations for sensitivity, specificity, and DOR are presented in Table 3. Range of sensitivity for 4 hospitals was 0.64 to 1.00, specificity for 3 hospitals was 0.50 to 0.75, and DOR for all sites was 4.73 to 7. Although sensitivity at 1 site was high (0.84), specificity was low (0.37). The logistic regression analysis is shown in Table 4. For the purpose of this analysis, the medication list on the BHHRFA was collapsed into ‘‘medication’’ and ‘‘no medication.’’ The model
265
Copyright © 2014 Wolters Kluwer Health | Lippincott Williams & Wilkins. Unauthorized reproduction of this article is prohibited.
Figure 1. The BHHRFA. Reprinted with permission.
266
JONA Vol. 44, No. 5 May 2014
Copyright © 2014 Wolters Kluwer Health | Lippincott Williams & Wilkins. Unauthorized reproduction of this article is prohibited.
Table 2. Age, Gender, and Nursing Shift When Data Were Collected for All Hospital Sites Age, Mean (SD) Baptist Health Hospital BHLex BHLexVICU Hospital A Hospital B Hospital C All hospitals (medical-surgical)
65.50 65.31 65.07 65.90 68.06 65.91
Gender (Female), % (n)
(16.06) (13.73) (17.03) (16.72) (15.52) (16.08)
56.8 (91,019) 45.4 (12,071) 53.9 (3,210) 60.5 (9,515) 57.6 (19,019) 57.1 (122,763)
that includes has fallen in the last 6 months, age, mental status, elimination, mobility, medications, and nurse’s clinical judgment was statistically significant (# 2[N = 214,998] = 329.37, P G .001). Together, these variables are able to distinguish between patients who would and would not fall. Five of the independent variables appeared to make a unique statistically significant contribution to the model (falls in the past 6 months, age, mental status, elimination, and nurses’ judgment). The strongest predictor of falls was ‘‘has fallen in the last 6 months’’ (odds ratio, 2.98). This value indicates that patients who have fallen in the past 6 months are approximately 3 times more likely to fall than those who have not fallen in the past 6 months, controlling for other factors in the model. Time taken for nurses (n = 134) to complete the BHHRFA ranged from 0.33 seconds to 3.00 minutes, with a mean (SD) time of 1.19 (0.56) minutes. Nurses taking more than 1 minute were new graduates. Critical care nurses (n = 20) took less time to complete the assessment (mean [SD], 0.63 [0.15] minutes) than medical-surgical nurses (n = 114) (mean [SD], 1.29 [0.54] minutes). Differences in volume and acuity of patients in medical-surgical and critical care areas may have contributed to this difference in time.
Discussion Age, gender, and the shift when assessments were administered were approximately the same across study
Shift am, % (n) 53.2 (85,284) 50.0 (13,302) 49.8 (2,987) 51.0 (8,015) 51.0 (16,819) 52.6 (113,105)
Shift pm, % (n) 46.8 (74,974) 50.0 (13,298) 50.2 (3,017) 49.0 (7,705) 49.0 (16,181) 47.4 (101,877)
sites. Results of data analyses show that the BHHRFA has strong sensitivity and DORs across 4 settings. Specificity was strong across 3 of the settings and low at 1 hospital (hospital C). A low specificity value at 1 site may be related to nurse behaviors in administering the BHHRFA. For example, an increasing elderly presence in acute care settings who have complex problems and therefore appear to be at higher risk for falling may lead nurses to rate patients at a higher risk for falls than necessary. They may have a bias regarding a particular patient population as likely to fall, and therefore, their ability to predict those who will not fall is low. In summary, regression analyses show that the model used in the development of this instrument is effective in terms of identifying those individuals who will fall. In relation to clinical usefulness, conducting this assessment takes nurses approximately 1 minute. Anecdotal data suggest that variation in time for nurses to complete the BHHRFA may be due to a nurse’s level of experience. For example, a more experienced nurse may take less time to complete the assessment than a new graduate. Limitations Although 4 sites (2 urban, 2 rural) were used to test the BHHRFA, all institutions were within 1 state and are part of the same corporation. Hospitals in different geographic locations within other corporate structures could find the BHHRFA more or less useful. In addition, this assessment was limited to adults; patients
Table 3. Sensitivity, Specificity, and DOR for the BHHRFA for All Hospital Sites Study Site
Hospital Sample Size (n)
BHLex BHLexVICU Hospital A Hospital B Hospital C All hospitals (medical-surgical)
160,274 26,601 6,004 15,720 33,000 214,998
Sensitivity (95% CI) 0.64 0.73 1.00 0.88 0.84 0.70
(0.58-0.71) (0.43-0.90) (0.51-1.00) (0.53-0.98) (0.74-0.91) (0.65-0.75)
Specificity (95% CI) 0.75 0.67 0.58 0.50 0.37 0.66
(0.74-0.75) (0.66-0.67) (0.57-0.60) (0.49-0.51) (0.37-0.38) (0.66-0.67)
DOR (95% CI) 5.29 (3.97-7.04) 5.34 (1.42-20.13) N/Aa 7.0 (0.86-56.91) 3.19 (1.68-6.10) 4.73 (3.70-6.10)
Abbreviation: CI, confidence interval. a DOR not available because of a sensitivity value of 1.
JONA Vol. 44, No. 5 May 2014
267
Copyright © 2014 Wolters Kluwer Health | Lippincott Williams & Wilkins. Unauthorized reproduction of this article is prohibited.
Table 4. Logistic Regression Predicting Likelihood of Falls in an Acute Care Setting Among Medical-Surgical Patients at All Sites (N = 214,998) Predictor Fall in the past 6 months Age group Mental status Elimination Mobility issue Medication Nurse’s judgment (0-10)
B a
1.09 j0.60a 0.77a 0.55a 0.18 0.36 0.19a
SE
Wald
df
Odds Ratio
95% CI
0.13 0.13 0.14 0.13 0.18 0.16 0.03
72.98 21.96 31.65 16.95 1.01 5.23 54.20
1 1 1 1 1 1 1
2.98 .55 2.16 1.73 1.20 1.43 1.20
2.32-3.83 0.43-0.71 1.65-2.82 1.33-2.25 0.84-1.71 1.05-1.93 1.15-1.26
Abbreviation: CI, confidence interval. a P G .001.
younger than 18 years were not included in the study. Only aggregate data were recorded and analyzed. Patients were assessed twice a day during their hospital admission and no attempt was made to differentiate multiple assessments or falls of individual patients.
attend to assessment scores and intervene with those patients deemed likely to fall. Numerous interventions designed to prevent falls are suggested in the literature and need to be tailored to the specific institution and patient population.
Conclusions and Implications for Practice
Recommendations
The development of the BHHRFA spanned a 5-year period. The 2 pilot projects as well as a continuing and comprehensive review of the literature formed the basis for the instrument. The collection of data across multiple sites and the size of the sample are strengths of this project. Although use of an assessment to predict falls is a crucial component of a falls prevention program, prediction alone is not sufficient to diminish the incidence of falls. Healthcare professionals must
Based on the outcomes of this study, further testing of the BHHRFA in different geographic locations and corporate structures is recommended. Positive outcomes regarding sensitivity, specificity, ease of use, and DOR analyses suggest use of this assessment with inpatients in acute care settings. Based on logistic regression analyses, a shorter version of the assessment might be possible. Continued testing of the BHHRFA is recommended.
References 1. Milisen K, Staelens N, Schwendimann R, et al. Fall prediction in inpatients by bedside nurses using the St. Thomas’s Rick Assessment Tool in falling elderly inpatients (STRATIFY) instrument: a multicenter study. J Am Geriatr Soc. 2007;55:725-733. 2. Englander F, Hodson TJ, Terregrossa RA. Economic dimensions of slip and fall injuries. J Forensic Sci. 1996;41(5):733-746. 3. Stevens JA, Mack KA, Paulozzi LJ, Ballesteros MF. Selfreported falls and fall-related injuries among persons aged Q65 yearsVUnited States, 2006. J Saf Res. 2008;39:345-349. 4. Rutledge DN, Donaldson NE, Pravikoff DS. Update 2003: fall risk assessment and prevention in hospitalized patients. Online J Crit Innov. 2003;6(5):1-55. 5. McAlister S. APR DRG weights and the relationship to patient falls. Nurs Econ. 2009;27(2):119-123. 6. Kim EA, Mordiffi SZ, Bee WH, Devi K, Evans D. Evaluation of three fall-risk assessment tools in an acute care setting. J Adv Nurs. 2007;60(4):427-435. doi:10.1111/j.1365-2648.2007.04419.x 7. Morse JM, Morse RM, Tylko SJ. Development of a scale to identify the fall-prone patient. Can J Aging. 1989;8(4):366-377. doi:10.1017/S0714980800008576 8. Oliver D, Britton M, Seed P, et al. Development and evaluation of an evidence based risk assessment tool (STRATIFY) to
268
9.
10.
11.
12. 13.
14.
15.
predict which elderly inpatients will fall: case-control and cohort studies. Br Med J. 1997;315:1049-1053. Hendrich A, Bender P, Nyhuis A. Validation of the Hendrich II Risk Model: a large concurrent case/control study of hospitalized patients. Appl Nurs Res. 2003;16(1):9-21. Aranda-Gallardo M, Morales-Asencio JM, Canca-Sanchez JC, et al. Instrument for assessing the risk of falls in acute hospitalized patients: a systematic review and meta-analysis. BMC Health Serv Res. 2013;13:122. Ivziku D, Matarese M, Pedone C. Predictive validity of the Hendrich fall risk model II in an acute geriatric unit. Int J Nurs Stud. 2011;48:468-474. Morse J. Computerized evaluation of a scale to identify the fallprone patient. Can J Public Health. 1986;77:21-25. Vassallo M, Stockdale R, Sharma J, et al. A comparative study of the use of four fall risk assessment tools on acute medical wards. J Am Geriatr Soc. 2005;53:1034-1038. Rauch K, Balascio J, Gilbert P. Excellence in action: developing and implementing a fall prevention program. J Healthc Qual. 2009;31(1):36-42. Meyer G, Kopke S, Haastert B, et al. Comparison of a fall risk assessment tool with nurses’ clinical judgment alone: a
JONA Vol. 44, No. 5 May 2014
Copyright © 2014 Wolters Kluwer Health | Lippincott Williams & Wilkins. Unauthorized reproduction of this article is prohibited.
cluster-randomized controlled trial. Age Ageing. 2009;38(4): 417-423. 16. Myers H, Nikoletti S. Fall risk assessment: a prospective investigation of nurses’ clinical judgment and risk assessment tools in predicting patient falls. Int J Nurs Pract. 2003;9(3): 158-165. 17. Choi YS, Lawler E, Boenecke CA, Ponatoski ER, Zimring CM. Developing a multi-systemic fall prevention model, incorporat-
JONA Vol. 44, No. 5 May 2014
ing the physical environment, the care process and technology: a systematic review. J Adv Nurs. 2011;67(12):2501-2524. doi:10 .1111/j.1365-2648.2011.05672.x 18. Inouye SK, van Dyck CH, Alessi CA, et al. Clarifying confusion: the confusion assessment method a new method of detection of delirium. Ann Intern Med. 1990;113(12):941-948. 19. IBM Corp. IBM SPSS Statistics for Windows, Version 21.0. Armonk, NY: IBM Corp; 2012.
269
Copyright © 2014 Wolters Kluwer Health | Lippincott Williams & Wilkins. Unauthorized reproduction of this article is prohibited.
