LWW/JNCQ

JNCQ-D-13-00102

May 21, 2014

18:12

J Nurs Care Qual Vol. 29, No. 3, pp. 263–268 c 2014 Wolters Kluwer Health | Lippincott Williams & Wilkins Copyright 

Reliability and Validity of the Postepidural Fall Risk Assessment Score Kathleen Thompson, PhD, CNE, RN; Lisa Haddad, PhD, RN; Sarah Smith, BSN, RN The purpose of this study was to establish the interrater reliability and face validity of the Postepidural Fall Risk Assessment Score instrument for the obstetric patient. The sample consisted of 207 healthy mothers at an inner-city level 1 trauma center. Kappa coefficients ranged from 0.54 to 0.83. Face validity of the tool was determined by participating nurses (n = 25). Results indicated that the tool was reliable and required modification to increase face validity. Key words: epidural, fall prevention, fall risk assessment, obstetric patient falls, Postepidural Fall Risk Assessment

P

REVENTION OF FALLS continues to be a major focus of maintaining patient safety. The National Patient Safety Goals established by The Joint Commission require hospitals to reduce falls and require institutions to identify appropriate preventive measures for the population and setting.1 To achieve the goal of reducing falls, several instruments have been developed to identify patients at risk of falling for the purpose of triggering preventive interventions. Several studies have evaluated the current tools available for assessing fall risk.2-6 However, these studies or reviews were designed to establish reliability and validity us-

Author Affiliations: College of Nursing, University of Tennessee, Knoxville (Dr Thompson); and 3East/Mother Baby (Ms Smith), Patient Services (Dr Haddad), University of Tennessee Medical Center, Knoxville. No outside funding for this study was obtained. The authors declare no conflict of interest. Correspondence: Lisa Haddad, PhD, RN, Patient Services, University of Tennessee Medical Center, 1024 Alcoa Hwy, Box 104, Knoxville, TN 37920 (lhaddad@ utmck.edu). Accepted for publication: November 27, 2013 Published ahead of print: December 26, 2013 DOI: 10.1097/NCQ.0000000000000043

ing the adult medical-surgical patient or more specifically the older adult. Although high fall rates are not common in obstetric units, patients who receive epidural anesthesia for pain control present a special population at risk for falls. These patients are usually healthy women who typically have the ability to ambulate without difficulty. Following an epidural, these otherwise healthy women are temporarily without the use of their lower extremities and may have sustained significant blood loss. They still believe, however, that they have the ability to ambulate safely. With the widespread use of epidural anesthesia during labor and delivery, mothers are at an increased risk for numbness, hypotension, nausea, and vomiting.7,8 The risk of hypotension is further increased with an average blood loss of 500 mL during vaginal delivery and 1000 mL during cesarean delivery.8 Identification of patients at an increased risk for falls is mandatory for any unit. For the obstetric unit, screening is especially important before ambulating the postepidural patient. At a southern regional medical center, patients are usually transferred to the motherbaby unit within 2 hours after delivery, with orders to ambulate the patient immediately for vaginal delivery and 6 to 8 hours following cesarean delivery. The nurse needs to 263

Copyright © 2014 Lippincott Williams & Wilkins. Unauthorized reproduction of this article is prohibited.

LWW/JNCQ

264

JNCQ-D-13-00102

May 21, 2014

18:12

JOURNAL OF NURSING CARE QUALITY/JULY–SEPTEMBER 2014

determine when it is safe for the patient to ambulate. Our unit did not have a standardized evaluation tool, and nurses identified a need for a measure to assess patients for any increased risk of falls. A review of the literature identified only one tool specifically to assess the postepidural obstetric patient, the Postepidural Fall Risk Assessment Score (PEFRAS).9 Auger and Gingras10 have conducted a study using the Dionne Egress test to determine a postpartum patient’s ability to ambulate safely. The test focuses on the patient’s ability to move, including from a sitting to standing position, marching in place, and stepping forward and backward, similar to items on the PEFRAS. The PEFRAS addresses the specific needs of the obstetric patient, modifying aspects of 2 tools commonly used to assess the adult medical-surgical patient: the Morse Fall Scale11 and the Modified Aldrete Score.12 Frank et al9 also incorporated the experiences of obstetric nurses. While Frank et al described the process used to develop the PEFRAS, they did not test the tool for reliability. They provided a rationale for each of the items as face validity of the instrument and recommended that further research be done to establish the reliability and validity of the instrument. The purpose of this methodological study was to test the psychometric properties of the PEFRAS and report the interrater reliability and face validity. Polit and Beck13 define interrater reliability as the degree to which 2 raters independently assign the same rating to an attribute being measured or observed. They define face validity as the extent to which the instrument measures what it actually purports to measure.13 METHODS The overall goal of this study was to establish the interrater reliability of the PEFRAS and further examine the content validity of the items. The study was conducted over a 3-month period. Permission to conduct the study was obtained from the hospital’s insti-

tutional review board. Frank et al9 granted permission to use the PEFRAS. Setting The study was conducted on a 30-bed mother-baby unit of a southern inner-city level 1 trauma hospital that has approximately 2500 births per year. The current hospital policy for vaginal delivery is for nurses to assess the obstetric patient on admission to the unit and ambulate the patient when the nurse and the patient determine it can be done safely. For cesarean delivery, the nurse assesses the obstetric patient, but the patient remains on bed rest for 4 to 6 hours, with ambulation 6 to 8 hours after delivery. The current policy also requires the nurse to complete an assessment as needed and assess the bleeding and fundal checks every 4 hours for the first 24 hours. Sample Sample size was determined using the formula for Cohen κ described by Lee.14 Setting α = .05 and β = .2, the required sample size was 157. A convenience sampling plan identified all patients admitted to the unit during a 90-day period meeting the inclusion criteria. Inclusion criteria included all postpartum patients who had an epidural, with stable vital signs, and without respiratory distress or other complications that would not allow them to ambulate. The only exclusion criterion was whether patients had received magnesium sulfate. Patients who received magnesium sulfate are considered unstable and are placed into a different fall risk category because of the drug being administered. A total of 67 patients were excluded from the study because they had received magnesium sulfate. The total sample comprised 207 patients. Patients could be rated more than once, and the total number of pair ratings was 357. No paired ratings were excluded because of ratings not being completed within the required time frame. Procedure Raters included all staff and charge nurses from the obstetric unit. All nurses underwent

Copyright © 2014 Lippincott Williams & Wilkins. Unauthorized reproduction of this article is prohibited.

LWW/JNCQ

JNCQ-D-13-00102

May 21, 2014

18:12

Reliability and Validity of the Postepidural Fall Risk Assessment Score a standard training session for using the PEFRAS. Every postepidural patient was rated using the PEFRAS on admission to the unit independently by both the assigned nurse and the charge nurse. All paired ratings occurred within 30 minutes of each other. Assessments using the scale were repeated independently by the staff nurse and the charge nurse with each ambulation attempt until the patient was able to ambulate independently. The timing of ratings following the initial assessment was variable, based on the patient’s condition. It took approximately 5 minutes for each practitioner to complete the PEFRAS. When oriented to the unit, nurses are assigned a mentor. When ambulating their patients, most nurses (n = 23; 92%) reported that they used the method demonstrated by their preceptors. However, no information regarding what this method entails was collected. They also indicated that they were more likely (n = 17; 74%) to ambulate the patient when the family was out of the room. Following completion of the rating phase of the study, the nurses (n = 25) were surveyed to assess their opinions about the instrument. They were asked whether the items were clear and concise, whether the nurses believed that all potential risks for falls were included, and whether they were comfortable using the results of the assessment in making the judgment to ambulate the patient or not. Measures The PEFRAS is an 8-item scale that consists of 7 yes/no questions and one 3-category response item. Frank et al9 described how each of the items represents fall risk factors that could be attributed to the postepidural obstetric patient. As discussed previously, relevant risk factors from previously developed tools for medical-surgical patients were incorporated including history of falls, medication administration, blood pressure history, preexisting conditions, and blood loss. Items specific to the obstetric patient were history of epidurals; hours since the epidural was discontinued; ability to lift legs, feet, and buttocks off the bed unassisted; and the stand

265

test. Yes/no responses were assigned either 0 point (low risk) or 20 points (high risk). The amount of time the epidural had been discontinued was assigned 0 point if more than 3 hours, 10 points if 2 to 3 hours, and 20 points if less than 2 hours. Total scores could range from 0 to 130. Although a rationale was not provided, Frank et al9 set a total score of more than 50 as indicative of an increased risk for fall. Analysis The analysis used both quantitative and qualitative methods. Quantitative analysis was conducted using SPSS (version 20; IBM Corp, Armonk, New York). To determine interrater reliability, κ coefficients were calculated. The κ coefficient represents the amount of agreement between 2 raters beyond what is expected to occur by chance alone.13 Face validity is determined by asking a set of experts to evaluate the items on the measure to provide their opinion about the adequacy of the items.13 The raters, experienced obstetric nurses, were asked a series of questions to determine the face validity of the instrument. RESULTS Twenty-five nurses participated in the study. Half of the nurses (n = 13) were younger than 30 years. Nursing experience ranged from 1 to 22 (M = 6.7, SD = 6.5) years, and experience in obstetrics ranged from 1 to 20 (M = 5.6, SD = 6.1) years. The majority (n = 18; 69%) held a baccalaureate degree, and the other 31% (n = 8) were prepared at the associate degree level. The 207 obstetric patients rated ranged in age from 14 to 42 (M = 26.4, SD = 6) years; this was the first or second delivery for the majority of the patients (n = 169; 77%); and the gestational age of the infants ranged from 25.2 to 42.1 (M = 38.5, SD = 2.1) weeks. Fall risk scores Overall risk scores ranged from 1 to 120 (M = 86, SD = 22), with 80% (n = 286) of the ratings above the 50-point cutoff for

Copyright © 2014 Lippincott Williams & Wilkins. Unauthorized reproduction of this article is prohibited.

LWW/JNCQ

JNCQ-D-13-00102

266

May 21, 2014

18:12

JOURNAL OF NURSING CARE QUALITY/JULY–SEPTEMBER 2014

risk established by Frank et al.9 One item on the scale was the “hours since the epidural turned off.” Scores could be more than 1, 2, or 3 hours, with lower scores assigned as the time from discontinuation increased. As anticipated, higher risk scores were associated with the timing of epidural discontinuation. Total scores for the 207 women rated within 2 hours of discontinuing the epidural ranged from 30 to 120 (M = 79, SD = 15.7). Ninety-five percent (n = 196) were rated 50 or above, indicating their high risk for falls. This percentage decreased to 65.5 (n = 136) if the epidural had been discontinued between 2 and 3 hours before rating and if the patient was rated after 3 hours. Only 18% of the patients (n = 37) were rated a 50 or higher. In addition to the timing of discontinuation, the 2 factors related to movement and activity were rated higher in the earlier time periods. As evident in Table 1, most women (n = 193; 90.5%) were not able to lift their lower extremities without assistance within 1 to 2 hours of the epidural being discontinued. This percentage decreased with time, with 62% (n = 81) and 46% (n = 13) of women able to lift their lower extremities at the >2- and

>3-hour time frames, respectively. The second item was ability to bend knees while standing without buckling. The percentage of women unable to successfully perform this maneuver decreased from 74% (n = 154) if less than 2 hours to 18.5% (n = 5) if more than 3 hours postepidural. Reliability analysis A total of 357 paired ratings of the bedside nurse and the charge nurse were used in the reliability analysis. Kappa coefficients were calculated for all pairs as well as the pairs based on the timing of the epidural discontinuation (>1, >2, and >3 hours). All of the κ coefficients were above the 0.40 parameter for establishing reliability.13 The κ coefficient for all pairs was 0.54. For ratings completed more than 1 hour after discontinuation, the κ coefficient was 0.83 and was 0.62 for ratings at both the >2- and >3-hour time periods. Validity analysis The best indicator of instrument validity would be a decrease in fall rates when the PEFRAS was instituted. No falls were reported during the course of the study. However,

Table 1. Frequency of Responses on the Postepidural Fall Risk Assessment Score at >1, >2, and >3 Hours Postepidural

Assessment Item Lifts lower extremities without assistance History of opioids History of unstable blood pressure History of preexisting illness EBL >500 mL for SVD or EBL >1000 mL for C/S Can bend knees while standing without buckling

Response Yes No Yes No Yes No Yes No Yes No Yes No

>1 h (n = 207), >2 h (n = 130), >3 h (n = 28), n (%) n (%) n (%) 20 (10) 187 (90) 99 (49) 106 (51) 20 (10) 187 (90) 34 (17) 173 (83) 11 (5) 196 (95) 50 (27) 151 (73)

49 (38) 81 (62) 59 (39) 80 (61) 13 (10) 117 (90) 19 (15) 111 (85) 5 (4) 125 (96) 78 (62) 47 (38)

15 (54) 13 (46) 7 (25) 21 (75) 5 (18) 23 (82) 4 (14) 24 (86) 2 (7) 26 (93) 22 (82) 5 (18)

Abbreviations: C/S, cesarean section; SVD, standard vaginal delivery; EBL, estimated blood loss.

Copyright © 2014 Lippincott Williams & Wilkins. Unauthorized reproduction of this article is prohibited.

LWW/JNCQ

JNCQ-D-13-00102

May 21, 2014

18:12

Reliability and Validity of the Postepidural Fall Risk Assessment Score given that there had only been 4 falls over the past 2 years, this change was not significant. As nurses participated in the study, they were asked several questions to assist in determining the face validity. Most nurses (n = 22; 88%) reported a high level of comfort using the results from the PEFRAS in making decisions about when their patient was able to ambulate safely. Nurses also were asked to provide feedback on the individual items of the PEFRAS. Most nurses voiced difficulty with finding the information in the medical record to rate 2 of the items: (1) “history of an epidural and/or fall” and (2) “hours since epidural turned off.” Item 4 did not have any variability, as all patients at this institution with an epidural receive fentanyl, an opioid. This item did not add to the ability of the instrument to discriminate. Most nurses indicated that item 3, “able to lift legs, feet and bottom off bed unassisted,” was a strong indicator of ability to ambulate. Comments about the overall effectiveness of the tool were primarily positive. Three nurses indicated that the use of the instrument increased their confidence in ambulating patients postepidural. The majority of negative comments related to the study’s methodology, specifically the need to coordinate ratings with the charge nurse, which was often time-consuming. There were 2 comments that the instrument was too long and needed to be shortened. These nurses believed that some patients were able to walk to the bathroom but were not allowed on the basis of the risk score above 50 that Frank et al9 established. DISCUSSION On the basis of κ values, results indicated that the PEFRAS as published was a reliable instrument for measuring a postepidural patient’s risk for fall. However, with the comments from the nurses, it was determined that the tool required modification and additional testing to improve the face validity. With permission, the original tool was modified after completion of the study. These modifications included both the elimination of some

267

items and the combination of others. Item 1 was changed to assist with clarity and read, “History of an epidural and or fall prior to this delivery.” Item 4 was changed to pain medicine after this delivery. Items 5 and 6 were combined to include both history of unstable blood pressure and preexisting illness. On the basis of the comments related to the assignment of high numbers to items, the assignment of scores for each of the items was recommended to be modified so that all items with yes/no responses would be scored 1 and 0, respectively. Total scores would now range from 0 and 8 (Table 2), with a total score of 5 or higher indicative of a high risk for falls. To address the concern by nurses with the difficulty in determining when the epidural had been discontinued, there was a change in the location of this information in the medical record by the labor and delivery nurse, allowing greater ease to find the needed information. The interrater reliability needs Table 2. Proposed Modifications to Items on the Postepidural Fall Risk Assessment Scorea Item

Scoring

History of an epidural and/or fall prior to this delivery Hours since epidural turned off

No = 0 Yes = 1 >3 = 0 >2 = 1 >1 = 2 No = 0 Yes = 1 No = 0 Yes = 1 No = 0 Yes = 1

Able to lift legs, feet, and buttocks off bed unassisted Pain medication administered after delivery Change in BP (>20 mm Hg) or a history of diabetes and/or preeclampsia EBL >500 mL for SVD and 1000 mL for C/S Test stand (at bedside): can bend knees without buckling

No = 0 Yes = 1 No = 0 Yes = 1

Abbreviations: BP, blood pressure; C/S, cesarean section; EBL, estimated blood loss; SVD, standard vaginal delivery. a If the score is more than 5, do not ambulate the patient.

Copyright © 2014 Lippincott Williams & Wilkins. Unauthorized reproduction of this article is prohibited.

LWW/JNCQ

JNCQ-D-13-00102

268

May 21, 2014

18:12

JOURNAL OF NURSING CARE QUALITY/JULY–SEPTEMBER 2014

to be reassessed with the recommended modifications. Three of the 8 items appeared to account for determining the patient’s risk for falling. These were the length of time the epidural had been discontinued, ability to move the lower extremities, and ability to bend knees without buckling. Ability to move is an important factor and provides additional rationale for the Dionne Egress test being evaluated currently.10 As described, this tool focuses primarily on the ability to move. Limitations This study was conducted with 1 sample of postpartum patients in 1 institution. Additional testing of the psychometric properties should be conducted in additional institutions to further establish reliability. Frank et al9 set a score of more than 50 as the criterion that would establish patients at a high risk for falls but did not provide a

rationale. Further study is needed to establish whether this criterion is a valid indicator of a fall risk and to identify the appropriate cutoff point that would indicate a high risk for falls. With the modification of instrument scoring, a score of 5 or higher was established as the cutoff. The predictive validity of the instrument needs to be determined. CONCLUSIONS This study demonstrated that the original PEFRAS had interrater reliability for this institution’s patient population. On the basis of the recommendations of the nurse raters, several modifications to the instrument were recommended. Additional testing of the psychometric properties of the PEFRAS, including interrater reliability and predictive validity, is required to establish the psychometric properties of the instrument with the recommended modifications.

REFERENCES 1. National Patient Safety Goals. The Joint Commission Web site. http://www.jointcommission.org/ patientsafety/nationalpatientsafetygoals. Published 2011. Accessed December 2011. 2. Eagle DJ, Salama S, Whitman D, Evan LA, Ho E, Olde J. Comparison of three instruments in predicting accidental falls in selected inpatients in general teaching hospital. J Gerontol Nurs. 1999;25(7): 40-45. 3. Kim EAN, Mordiffi SZ, Bee WH, Devi K, Evans D. Evaluation of three fall-risk assessment tools in an acute care setting. J Adv Nurs. 2007;60(4):427435. 4. Gates S, Smith L, Fisher JD, Lamb SE. Systematic review of accuracy of screening instruments for predicting fall risk among independently living adults. J Rehabil Res Dev . 2008;45(8):1105-1116. 5. Kehinde JO. Instruments for measuring fall risk in older adults living in long-term care facilities: an integrated review. J Gerontol Nurs. 2009;35(10): 46-55. 6. Poe SS, Cvach M, Dawson PB, Straus H, Hill EE. The John Hopkins Fall Risk Assessment Tool: postimplementation evaluation. J Nurs Care Qual. 2007;22(4):293-298. 7. Hawkins JL. Epidural analgesia for labor and delivery. N Engl J Med. 2010;362(16):1503-1510.

8. Pillitteri A. Maternal and Child Health Nursing: Care of the Childbearing and Childrearing Family. 6th ed. Philadelphia, PA: Lippincott Williams & Wilkins; 2010. 9. Frank B J, Lane C, Hokanson H. Designing a postepidural fall risk assessment score for the obstetric patient. J Nurs Care Qual. 2009;24(1):50-54. 10. Auger JE, Gingras D A. Fall risk prevention in postpartum patients. J Obstet Gynecol Neonatal Nurs. 2012;41(suppl 1):S12-S13. 11. Morse JM, Black C, Oberle K, Donahue P. A prospective study to identify the fall-prone patient. Soc Sci Med. 1998;28(1):81-86. 12. Aldrete JA. Modifications to the post anesthesia score for use in ambulatory surgery. J Perianesth Nurs . 1998;12(3):148-155. 13. Polit DF, Beck CB. Nursing Research: Generating and Assessing Evidence for Nursing Practice. 9th ed. Philadelphia, PA: Lippincott Williams & Wilkins; 2012. 14. Lee T. On Determination of Sample Size for the Positive Kappa Coefficient. Joint Statistical Meetings—Statistical Computing Section. Atlanta, GA: Centers for Disease Control and Prevention. http://www.amstat.org/sections/SRMS/ proceedings/y2002/files/JSM2002-001144.pdf. Accessed December 2011.

Copyright © 2014 Lippincott Williams & Wilkins. Unauthorized reproduction of this article is prohibited.

Reliability and validity of the postepidural fall risk assessment score.

The purpose of this study was to establish the interrater reliability and face validity of the postepidural fall risk assessment score instrument for ...
93KB Sizes 1 Downloads 0 Views