International Journal of Nursing Studies 52 (2015) 666–676

Contents lists available at ScienceDirect

International Journal of Nursing Studies journal homepage: www.elsevier.com/ijns

The impact of an emergency department nursing intervention on continuity of care, self-care capacities and psychological symptoms: Secondary outcomes of a randomized controlled trial Sylvie Cossette a,*, Nancy Frasure-Smith b, Alain Vadeboncoeur c, Jane McCusker d, Marie-Claude Guertin e a

Faculty of Nursing, University of Montreal, Montreal Heart Institute Research Center, Canada Psychiatry and Nursing, McGill University, Centre hospitalier de l’Universite´ de Montre´al and Montreal Heart Institute Research Center, Canada c Emergency Department, Montreal Heart Institute, Canada d Department of Epidemiology, Biostatistics, and Occupational Health, McGill University, St. Mary’s Research Centre, Canada e Biostatistics, Montreal Health Innovations Coordinating Center, Canada b

A R T I C L E I N F O

A B S T R A C T

Article history: Received 27 June 2014 Received in revised form 11 December 2014 Accepted 19 December 2014

Background: As yet there is no firm evidence about the types of intervention that can reduce

Keywords: Clinical trials Randomized Quality of health care Nursing care Emergencies Heart diseases Health services utilization

emergency room revisits. However, the literature on emergency room revisits suggests patient difficulties with managing their health problems and treatments after discharge may play a role. Objectives: We carried out a randomized trial of an emergency department-based nursing intervention, but results showed no reduction in revisits (primary outcome). This paper describes the secondary outcomes of the trial: patient perceptions of continuity of care, illness perceptions, self-care capacities, psychological symptoms and medication adherence 30 days after emergency room discharge. Design: Randomized, controlled trial. Settings: The trial was conducted in the emergency department of a tertiary cardiac hospital in Montreal, Canada between November 2007 and March 2010. Participants: The study involved 203 patients, including 108 in the experimental group and 95 in the control group. Methods: The intervention included one nurse patient encounter before discharge and two phone calls in the 10 days after discharge. Participants provided data 30 days post-discharge on secondary outcomes potentially related to emergency department revisits. Results: Although, as previously reported, the intervention had no impact on the primary outcome of emergency department revisits, the present study demonstrated a significant positive effect on patients’ perceived continuity of care (p = .033), self-care capacities (p = .037), anxiety (p = .007) and depressive symptoms (p = .043), and the illness perceptions treatment control subscale (p = .037). No differences were found for other illness perception subscales or medication adherence (all p’s > .05). Conclusion: Although the intervention did not influence emergency department revisits it did improve secondary outcomes, suggesting pathways for future research.

ß 2015 Elsevier Ltd. All rights reserved.

* Corresponding author at: Montreal Heart Institute Research Center, S-2510, 5000, Be´langer est, Montreal (Quebec), Canada H1T 1C8. Tel.: +1 514 376 3330x4012; fax: +1 514 793 7441. E-mail address: [email protected] (S. Cossette). http://dx.doi.org/10.1016/j.ijnurstu.2014.12.007 0020-7489/ß 2015 Elsevier Ltd. All rights reserved.

S. Cossette et al. / International Journal of Nursing Studies 52 (2015) 666–676

What is already known about the topic?  Frequent, unnecessary use of the emergency department causes overcrowding and delays that may jeopardize outcomes for patients who need urgent care.  Most interventions aimed at reducing unnecessary emergency department visits have not been successful.  Because of their potential link with health service utilization, secondary outcomes such as perceived continuity of care, illness perceptions, self-care capacities, psychological symptoms and medication adherence might influence ED revisits. What this paper adds  Several secondary outcomes were improved by an emergency department-based nursing intervention: continuity of care, self-care capabilities, anxiety and depressive symptoms and one aspect of illness perceptions.  Improving these secondary outcome variables was not sufficient to reduce emergency department revisits. The emergency department is a major entry point into the health care system of many countries with around onethird of individuals aged 15 or more reporting having visited an emergency department in the last two years in Canada (38%), the United States (34%), Australia (29%), the United Kingdom (29%), and New Zealand (27%) (Canadian Institute of Health Information, 2005; Schoen et al., 2004). Unnecessary emergency department revisits may result in overcrowding, increased waiting time, and failure to provide appropriate emergency care. We developed an emergency department-based intervention aimed at reducing emergency department revisits by targeting secondary outcomes that may predict emergency department utilisation according to the Andersen (1995) model of service utilization. The aim of the present paper is to report the impact of the intervention on the secondary outcomes of perceived continuity of care, illness perceptions, self-care capacities, psychological symptoms and medication adherence. 1. Background The emergency department-based interventions literature focuses primarily on service use and ways to reduce emergency department revisits, with very little focus on impacting secondary outcomes, i.e. those factors that may influence emergency department utilisation. Systematic reviews of clinical emergency department-based interventions (Althaus et al., 2011; Fealy et al., 2009; McCusker and Verdon, 2006) have summarized the results of eight randomized controlled trials conducted in Canada (Gagnon et al., 1999; Lang et al., 2006; McCusker et al., 2003), United States (Mion et al., 2003; Shumway et al., 2008; Spillane et al., 1997), Australia (Caplan et al., 2004) and Sweden (Hansagi et al., 2008). Only one trial reported a significant impact on both secondary outcomes and revisits (Shumway et al., 2008). Shumway et al. found that an intervention delivered by a social worker to frequent emergency department users with psychosocial problems

667

improved secondary outcomes such as peer and social service support, while also reducing emergency department revisits. Two other emergency department-based intervention studies observed some impact on secondary outcomes but did not observe effects on emergency department revisits (McCusker et al., 2003; Mion et al., 2003). McCusker et al. showed that an intervention in high-risk seniors was effective in improving secondary outcomes such as increased use of homecare services, increased primary physician referral and reduced 4-month functional decline. However, only one-third of patients who were referred to their primary care physician actually went to an appointment during the month after the emergency department visit. Therefore, failure to increase primary care use may have contributed to the intervention’s lack of impact on emergency department revisits. Mion et al. (2003) found that a comprehensive geriatric assessment of seniors in the emergency department resulted in higher satisfaction with the information they received. However, this positive impact on a secondary outcome was not accompanied by a reduction in emergency department revisits. Gagnon et al. (1999) found no impact of a 10-month nurse case management intervention for frail elderly patients on secondary outcomes. There was also an unexpected greater mean number of emergency department visits in the intervention group than in the usual care group. Two other emergency department-based randomized controlled trials reported no impact of interventions on either secondary outcomes or emergency department revisits (Caplan et al., 2004; Spillane et al., 1997). Other studies tested non-clinical interventions to reduce emergency department revisits. Electronic linkage between the emergency department and family physicians in both general Canadian (Lang et al., 2006) and Swedish populations (Hansagi et al., 2008) showed no difference in emergency department revisits. However, positive secondary outcomes were found in Hangasi et al., i.e. family physicians judged the information received from the ED useful and valuable. It is difficult to pinpoint the key to success among interventions that have had an impact on emergency department revisits or secondary outcomes. While all interventions used models of case management combined with screening for at-risk patients, they tended to be individualized to the particular needs of the patients, with variable intensity and duration. Interventions lasting up to 10 months (Gagnon et al., 1999) were no more successful than single contact interventions (McCusker et al., 2003; Mion et al., 2003). Recently, we reported the primary outcome results of a randomized controlled trial of an intervention delivered in the emergency department of a tertiary cardiac hospital (Cossette et al., 2013). At 30 days post-discharge, emergency department revisit rates were similar in the experimental and control groups (18% vs. 20% respectively, p = .81), with similar patterns seen at 90 days (p = .44), 180 days (p = .98) and 365 days (p = .75). As mentioned in the trial registration protocol (Current Controlled Trials, 2008) (ISRCTN88422298) we also selected specific secondary outcomes based on Andersen’s model of service

668

S. Cossette et al. / International Journal of Nursing Studies 52 (2015) 666–676

utilization (1995) that includes predisposing, need and enabling factors that may be related to emergency department revisits. The analysis presented in this paper focuses on these secondary outcomes. Illness perceptions and psychological symptoms were considered predisposing factors as they reflect the patient’s beliefs about the illness, and psychological characteristics that may predispose to service use. Self-care capacities and medication adherence were seen as need factors, considered the prime determinant of service utilization. According to Andersen, enabling factors refer to availability of the service, the users’ knowledge of the service and to the quality of interpersonal relationships that can facilitate or impede the utilization of the service. Patients’ perception of continuity of care was, thus, considered an enabling factor and a measure of quality of care.

2. Purpose The goal of the present trial was to evaluate the effectiveness of an emergency department-based nursing intervention. The primary outcome was a reduction in ED revisits (Cossette et al., 2013). The secondary outcomes, presented in this paper, include patients’ perceptions of continuity of care, illness perceptions, self-care capacities, psychological symptoms and medication adherence 30 days after discharge from the emergency department of a tertiary cardiac hospital.

3. Methods 3.1. Design We conducted a randomized controlled trial to compare the emergency department-based intervention with usual care. The protocol was reviewed and approved by the Research Ethics Board of the hospital, each participant signed a written consent before participating, and CONSORT guidelines were followed throughout (Zwarenstein et al., 2008). 3.2. Study setting and participants The study was conducted in adult patients at the emergency department of a tertiary cardiac hospital in Montreal, Canada. To be eligible, patients ready for discharge from the emergency department had to be atrisk for emergency department return based on two criteria identified in a previous study: at least one emergency department visit during the year prior to the initial visit, and current treatment with at least six medications (Vadeboncoeur et al., 2003). Exclusion criteria included inability to speak French or English, and cognitive problems (e.g. dementia) that would preclude provision of informed consent either noted in the medical chart or identified based on the clinical judgment of the project nurse To avoid multiple interveners for the same patient, we also excluded patients already receiving other regular follow-up (e.g. at a specialized clinic in the hospital or from external resources).

Potential participants received usual care from their regular bedside nurse until the emergency department medical discharge signature was obtained and discharge information was given by the bedside nurse. The study was explained to eligible patients and after they gave informed consent, a self-report questionnaire was administered to collect baseline data before discharge. All patients responded to the sociodemographic questionnaire. However, the baseline questionnaires on secondary outcomes were not mandatory for inclusion in the randomized controlled trial because patients leaving the emergency department are often rushed, and have family members waiting. 3.3. Interventions In the usual care control group, the project nurse repeated the advice already given by the bedside nurse that patients should contact regular healthcare resources such as telephone health hotlines, family physicians, cardiologists, or emergency services as needed after discharge. No specific intervention was provided to the control group in order to assure that their care was as similar as possible to the usual care in the emergency department. We developed the intervention for the experimental group based on Ashton and Wray’s (1996) proposition that, to avoid unscheduled emergency department revisits, clinical stability should be assured prior to emergency department discharge, and patients should be prepared to deal with potential post-discharge concerns. For the sake of parsimony and future transfer to clinical practice, and because there was no clear evidence suggesting that a longer intervention would be more powerful than a shorter one, we developed a short-term intervention that included three encounters: one at discharge, and two telephone follow-ups at 2–4 days and 7–10 days post-discharge. The intervention provided by a project nurse was individualized with the potential concerns of each patient assessed using a 19-item clinical disease management tool developed and refined in past studies with similar clients (Cossette et al., 2001, 2002; Frasure-Smith et al., 1997). The assessment evaluated patients’ capacities to cope with: (1) worries about readiness to return home; (2) disease and symptom management; (3) treatment management; (4) activities of daily living (ADL) and instrumental ADL management; (5) emotions and cognition; (6) informal resources; and (7) the health care system. For each of the 19 items patients were rated as ‘‘no risk’’, ‘‘presence of risk, but coping strategies in place’’, ‘‘at risk’’, or ‘‘not evaluated’’, as suggested by the work of McCusker et al. (2003). All items had to be evaluated, unless not clinically relevant. When a patient was rated as at-risk for any item, nurses’ interventions included: (1) teaching; (2) normalizing; (3) listening; (4) reassuring; (5) reframing; (6) confronting; (7) providing advice, recommendations; (8) warning; (9) giving positive feedback; (10) referring to external resources; and (11) reinforcing-external resources (e.g. increasing dosage or frequency of resource). After each encounter, the project nurse checked off which nursing intervention was retained in response to the specific

S. Cossette et al. / International Journal of Nursing Studies 52 (2015) 666–676

concerns expressed by patients. Because the intervention was individualized, each patient received a different intervention package. Patients were allowed to call the nurse between the planned encounters if they had any questions or concerns. Because the project nurses had access to the hospital chart, they were aware of emergency department visit characteristics including diagnosis, procedures and treatment, medications, discharge planning, and any other special issues – and could therefore personalize the intervention according to the patient’s clinical condition. Four project nurses worked on the project. All project nurses held a bachelor’s degree and had at least 5 years of experience in clinical cardiac care, though not necessarily in the emergency department. Minimal training was required to take part in the present study. 3.4. Outcomes measures The secondary outcomes were assessed with the Heart Continuity of Care Questionnaire (Hadjistavropoulos et al., 2004), the Illness Perception Questionnaire-Revised (Weinman et al., 1996), the Therapeutic Self-Care Tool (Doran et al., 2006; Sidani, 2003), the Hospital Anxiety and Depression Scale and the Self-Reported Medication-Taking Scale (Zigmond and Snaith, 1983). In addition to being assessed at baseline in patients who were able to fill out the questionnaires, these measures were readministered by telephone at 30 days post-discharge. The Heart Continuity of Care Questionnaire designed for cardiac patients (Hadjistavropoulos et al., 2004) includes 41 items yielding a total score, as well as three subscales. The informational continuity subscale assessed the patient’s perception of communication and information exchange between professionals, the patient, and the family during the emergency department stay. The relational continuity subscale assessed the link between hospital and community resources as well as patient satisfaction with postemergency department community care. The management continuity subscale assessed planned follow-up in the first month after emergency department discharge. The scale was translated from English to French using the back translation method. The authors also permitted adaptation of the wording from ‘‘doctor’’ to ‘‘health professionals’’ and from ‘‘cardiac problems’’ to ‘‘health problems’’. Patients indicated whether they strongly disagreed (1) to strongly agree (5) with each statement. Three subscales of the Illness Perception QuestionnaireRevised (Weinman et al., 1996) assessed the consequences of the illness (6 items), the perceived personal control (6 items), and the perceived treatment control (5 items). Patients indicated whether they strongly disagreed (1) to strongly agree (5) with each statement. The Therapeutic Self-Care Tool (Doran et al., 2006; Sidani, 2003) includes 12 items measuring actions taken by a patient to promote, maintain or improve health, prevent sickness, detect and manage symptoms, and regain normal functioning. Patients indicated whether they agreed with each statement from not at all (0) to totally (5). The Hospital Anxiety and Depression (HAD) Scale (Zigmond and Snaith, 1983) includes two subscales of

669

7 items each assessing anxiety and depressive symptoms, not severe pathological states. Each item is scored from zero to three and the total score for each subscale can range from zero to 21 for the continuous scores. The clinical scores are classified as follows: normal scores varying between zero and 7, borderline scores between 8 and 10 and abnormal scores between 11 and 21 on both subscales. Medication adherence was assessed using the 4-item Self-Reported Medication-Taking Scale (Morisky et al., 1986). Patients indicated whether (1) or not (0) they forgot (item 1), omitted (item 2), were careless (item 3), or stopped their medication when feeling better (item 4) at 30 days after discharge. In the present study, scores were dichotomized as 0 (never miss Rx), vs. 1 or more (1 missing Rx) because the distribution was skewed towards zero. In addition to the above outcome measures, we also analysed the concerns that were expressed by the experimental group during the intervention interviews at baseline, day 3, and day 10. Baseline socio-demographics and clinical variables were self-reported (including employment status, education, marital status, living alone, driving status, location of residence and whether or not the patient had a family physician) or were obtained from medical charts (including age, gender, diagnosis, emergency department stay characteristics, medication, external referrals at discharge). The Canadian Triage scale score (Bullard et al., 2008) included in the medical chart has five levels ranging from 1 (resuscitation) to 5 (non-urgent). 3.5. Sample size and randomization As recommended by the CONSORT Statement (http:// www.equator-network.org/reporting-guidelines/consort/ ), the sample size of 462 was based on the primary outcome of the trial, i.e. emergency department revisits, and calculated to achieve 80% power with a two-sided alpha of 0.05 to detect a reduction in revisits from 23% (baseline % of revisits based on administrative emergency department data) to 14% (based on the % reduction possible reported in the literature). No power analysis was performed for the secondary outcomes examined in the present paper. The randomization sequence was generated by an independent statistician from the Montreal Health Innovations Coordinating Center (MHICC), using the PROC PLAN procedure in SAS (SAS Institute Inc., Cary, NC, USA). The statistician provided opaque envelopes containing randomization assignments to the project nurse who was blinded to study group until opening the envelope. After the envelope was opened and the patient assigned to the intervention or usual care group, neither the nurse nor the participant were blind to the study group allocation. However, the research assistant who collected outcome measures data by telephone was blinded to study group assignment. 3.6. Data analysis Sociodemographic and clinical variables were summarized as means (SDs) for continuous variables and as

670

S. Cossette et al. / International Journal of Nursing Studies 52 (2015) 666–676

counts and percentages for categorical variables. Because the present study involved a sub-sample of the original randomized study we compared the baseline characteristics of the experimental and control groups using t-tests or Chi-square tests. The main analyses for the secondary outcomes expressed as continuous scores at 30 days were carried out using analysis of variance (ANOVA). Categorical outcomes at 30 days (e.g. clinical scores of depression and anxiety and medication adherence) were analyzed using Chi-square tests. To support the results we carried out two ancillary analyses. First, in patients with missing 30-day data, but with baseline data on the corresponding variable, we used the last value carried forward technique (Twisk and de Vente, 2002), using the baseline value to calculate the 30day score. This method is conservative in that it assumes that 30-day scores would be similar to baseline values, and not improved or diminished by the intervention. Second, separate analyses of covariance were performed to control for baseline characteristics that were thought to be clinically important based on the literature or that showed an imbalance between the experimental and control groups (p values < .10). Like the main analyses for the secondary outcomes, we did not control for corresponding baseline variables since these measures were not mandatory and therefore available for only half (N = 143, 75 EG and 68 CG) of the sample. A pre-planned mid-study interim analysis was performed by an independent committee of three experts in the fields of emergency department service-use, geriatric care and cardiac care. The interim statistical analysis based on the futility principle (Snapinn et al., 2006) was carried out by an independent statistician from the MHICC, and results were transmitted to the committee of experts which then provided recommendations to the research team. 4. Results 4.1. Reliability of the outcomes measures For the continuity of care questionnaire, alphas for the total score as well as the informational, relational, and management subscales were .88, .86, .65 and .77 respectively. For the IPQ-R, alphas for the consequences of the illness, the perceived personal control and treatment control were .77, .73 and .75 respectively. Alpha for the TSCT was .71 and alphas for the HAD-anxiety and HAD-depressive symptoms subscales were .73 and .78 respectively. 5. Interim analysis results The mid-study recruitment target of 231 of the planned 462 patients was reached in January 2010, but recruitment continued until the interim analysis was completed. On March 30, 2010, recruitment was stopped because early termination of the study was recommended by the independent committee based on the low likelihood of a significant impact on the primary outcome of emergency department revisits.

6. Participant flow Between November 13, 2007 and March 30, 2010 the project nurse evaluated 3795 patients (Fig. 1), of whom 2359 did not meet study criteria, 825 had logistical issues (e.g. discharge time outside of nurse’s shift) and 346 refused to participate. A total of 265 patients were randomized, including 132 to the experimental group and 133 to the control group. At the 30-day telephone follow-up, i.e. the timing of the secondary outcome assessment, 203 of the 265 original study patients were reached and 62 were lost to follow-up, and were therefore dropped from the main analyses. There were more losses to follow up in the control group (n = 38/ 133, 29%) than the experimental group (n = 24/132, 18%). Lost patients were less educated than retained patients (75% vs. 51% with high school education or less). All other baseline sociodemographic, clinical and emergency department characteristics were similar. 6.1. Characteristics of the sample with secondary outcome data The present sample was composed of the 203 patients (n = 108 in the experimental group and 95 in the control group) who provided 30-day data on the secondary outcomes. While the raw differences were small (see Tables 1–3), the experimental group was more likely to drive a car (p = .067), to have been transported to the emergency department by ambulance (p = .038), and took a lower mean number of medications at emergency department entry (p = .092). Although the difference between groups was not significant (p = .17), there was a greater proportion of males (61.1%) in the experimental group than in the control group (51.6%) and gender differences in cardiac patients may be important (Mosca et al., 2011). 6.2. Experimental group patients’ concerns during the nursing encounters In the experimental group, all 108 patients participated in the pre-discharge face-to-face encounter, while 104 participated in the first, and 106 in the second telephone follow-up. Each of the three encounters lasted from 5 to 60 min, but the pre-discharge encounter lasted a maximum of 25 min. As shown in Table 4, about half of the sample had at least one concern that was dealt with during the predischarge encounter, the most common being lack of control over the health problem (16.7%) and knowledge and understanding of the treatment (15.7%). During the first telephone call at 3 days post-discharge, concerns focused on consequences of the health problem (16.7%) and the management of clinical symptoms (14.8%). These concerns were less common at 10 days, but management of clinical symptoms was still a worry for 12% of the sample. Of note, emotional reactions were a source of concern for 5–11% of the experimental group at some point over the three encounters. Nursing interventions most often included education, advice, reassurance, and referrals, depending of the type of concerns.

S. Cossette et al. / International Journal of Nursing Studies 52 (2015) 666–676

Assessed for eligibility (n = 3795)

Enrollment

Randomized (n = 265)

671

Excluded (n = 3530) -Not meeting study criteria (n = 2359) - Not returning home, i.e. hospitalized, returning to an acute care hospital rehabilitation center, or long-term care (n = 998) - Not speaking English or French (n = 365) - Cognitive/psychological/problems including drug or alcohol problems, medical treatment refusal, severe physical comorbidities/ complications, death before discharge, hospitalized > 7 days (n = 251) - Other specialized clinical or research follow-up (n = 745) - Logistic reasons including hospital discharge outside the research nurse schedule (n = 825) - Refusal (n = 346)

Allocation Allocated to intervention (n = 132)

Allocated to usual care (n = 133)

Follow-Up Did not complete the 30 day process of care questionnaires (n = 24)

Did not complete the 30 day process of care questionnaires (n = 38)

Analysis Analysed (n = 108)

Analysed (n = 95)

Fig. 1. CONSORT flow diagram for the Transit-ED clinical trial secondary outcomes study. Table 1 Baseline sociodemographic characteristics. Characteristic

Experimental (n = 108)

Control (n = 95)

p value

Mean age (SD) Age 65 years old, n (%) Male, n (%) Employed, n (%) (n = 200) Education, n (%) high school (n = 198) Married, n (%) (n = 200) Lives alone, n (%) (n = 200) Drives a car, n (%) (n = 174) Lives on Montreal Island, n (%) Has family physician, n (%)

67.06 (10.42) 70 (64.8%) 66 (61.1%) 23 (21.9%) 56 (53.8%) 64 (59.3%) 32 (29.9%) 71 (80.7%) 72 (66.7%) 100 (92.6%)

67.33 (9.11) 58 (61.1%) 49 (51.6%) 22 (23.2%) 44 (46.8%) 58 (63.0%) 24 (25.8%) 59 (68.6%) 67 (70.5%) 85 (89.5%)

.85a (95% CI .58b .17b .83 b .32b .58b .52b .067b .55b .44b

a b

2.99, 2.47)

t-Test for independent sample. Chi-square test.

6.3. Impact of the intervention on secondary outcomes As shown in Table 5, the main analyses of variance showed that patients in the experimental group obtained better scores on the continuity of care variables (total score, and all three subscales). They also had better scores on the illness perceptions ‘‘perceived treatment control’’

subscale, although the raw mean difference was small. Therapeutic self-care capacity scores were higher in the experimental group, and anxiety and depression scores were lower. There were also fewer patients in the experimental group scoring within the ‘‘abnormal’’ range of the anxiety score at 30 days. No differences were found for medication adherence.

S. Cossette et al. / International Journal of Nursing Studies 52 (2015) 666–676

672 Table 2 Baseline clinical characteristics. Discharge diagnosis

Experimental (n = 108)

Control (n = 95)

Cardiac-related (any) Chest pain (stable angina, atypical chest pain, eliminate unstable angina) Any arrhythmia Heart failure Hypertension Post procedure complications (e.g. hematoma, pleural effusion) Problems related to medications Vascular

79 29 26 10 6 4 2 2

(73.1%) (26.9%) (24.1%) (9.3%) (5.6%) (3.7%) (1.9%) (1.9%)

68 23 29 5 2 3 3 3

(71.6%) (24.2%) (30.5%) (5.3%) (2.1%) (3.2%) (3.2%) (3.2%)

Non-cardiac related (any) Chronic obstructive pulmonary disease, asthma, or pneumonia Gastrointestinal (e.g. dyspepsia, gastroesophageal reflux) Psychological (e.g. anxiety, depression) Other Symptoms (e.g. dyspnea, lipothymia, anemia, falls, cold, cough)

29 2 4 3 20

(26.9%) (1.9%) (3.7%) (2.8%) (18.5%)

27 3 3 4 17

(28.4%) (3.2%) (3.2%) (4.2%) (17.9%)

Chi-square for discharge diagnosis (any cardiac related/any non-cardiac related) p = .80.

Table 3 Characteristics of the initial emergency department visit. Characteristic

Experimental (n = 108)

Control (n = 95)

p value

Arrival at the emergency department by ambulance (vs. other transportation), n (%)

23 (21.3%)

10 (10.5%)

.038b

Triage score, n (%) Resuscitation (1) Emergent (2) Urgent (3) Less urgent (4) Non urgent (5)

0 4 47 52 5

0 3 47 43 2

% on stretcher (vs. ambulatory) % visit the emergency department with someone (vs. no) % brought from home (vs. elsewhere) Number of hours in the emergency department (range 1,74), mean (SD) Number of medications on arrival (range 6,24), mean (SD)

75 (69.4%) 55 (50.9%) 92 (92.9%) 18.61 (15.81) 9.20 (2.79)

61 (64.2%) 41 (43.2%) 84 (91.3%) 20.40 (17.55) 9.95 (3.47)

.43b .27b .68b .45a (95% CI 6.41, 2.83) .092a (95% CI 1.61, .12)

Change of medication at discharge, n (%) No change (vs. any change) Dosage change (vs. no dosage change) Medication added (vs. no addition) Medication withdrawn (vs. not withdrawn)

39 34 49 24

33 28 42 18

(34.7%) (29.5%) (44.2%) (18.9%)

.84b .76b .87b .57b

External referrals at discharge To family physician (vs. no), n (%) Any other reference (MD specialist, lab or test (e.g. holter) (vs. no), n (%)

16 (14.8%) 63 (58.3%)

17 (17.9%) 58 (61.1%)

.55b .69b

a b

.69b (3.7%) (43.5%) (48.1%) (4.6%)

(36.1%) (31.5%) (45.4%) (22.2%)

(3.2%) (49.5%) (45.3%) (2.1%)

t-Test for independent sample. Chi-square test.

There were 7 experimental group and 15 control group patients with missing 30-day data, for whom we used their ‘‘last value carried forward’’ to replace the 30-day scores for a total of 115 and 110 patients analyzed in the ancillary analysis. Results of these analyses were similar to those in Table 5 with only small variations in p-values and means. Results were also similar after controlling for variables that were imbalanced between the treatment and control groups (e.g. driving a car vs. not, arrival at the emergency department by ambulance vs. other transportation, mean number of medications on arrival and gender). We did not control for all variables simultaneously as it reduced the available sample size from 203 to 165 because of missing data on one or more control variables.

7. Discussion The present analyses demonstrate that several secondary outcomes were improved including patients’ perception of continuity of care, illness perceptions (perceived treatment control subscale), therapeutic self-care capacities and anxiety and depression symptoms. There was no evidence of an impact on patient report of medication adherence. The current results are in line with the international literature. In the existing RCT literature only Shumway et al. (2008) showed an impact on both the primary outcome of emergency department revisits and on a secondary outcome. Three other studies which assessed secondary outcomes did not find hypothesized effects on either primary or secondary outcomes (Caplan et al., 2004;

S. Cossette et al. / International Journal of Nursing Studies 52 (2015) 666–676

673

Table 4 Experimental group patients’ concerns during the nursing encounters (n = 108). Nurse–patient encounters Baseline

Day 3

Day 10

1 sources of concerns, n (%)

1.26 (1.65) (range 0,8) 56 (51.9%)

1.14 (1.67) (range 0,7) 52 (48.1%)

0.99 (1.51) (range 0,6) 43 (39.8%)

Disease management Knowledge and understanding of the illness, n (%) Management of clinical symptoms, n (%) Perception of lack of control over the health problem, n (%) Perception of the consequences of the health problem, n (%)

13 14 18 11

(12.0%) (13.0%) (16.7%) (10.2%)

10 16 15 18

(9.3%) (14.8%) (13.9%) (16.7%)

8 13 9 6

(7.4%) (12.0%) (8.3%) (5.6%)

Treatment management Knowledge and understanding of the treatment, n (%) Treatment adherence, n (%) Self-administration of the treatment, n (%) Perception of control over the treatment, n (%)

17 7 7 6

(15.7%) (6.5%) (6.5%) (5.6%)

5 2 4 7

(4.6%) (1.9%) (2.0%) (3.4%)

8 3 2 11

(7.4%) (2.8%) (1.9%) (10.2%)

(1.9%) (4.6%) (3.7%) (1.9%)

6 9 3 3

(5.6%) (8.3%) (2.8%) (2.8%)

1 4 1 6

Number of concerns, mean (SD)

Life management Activity of daily living and Instrumental activity of daily living autonomy, n (%) Limitations in life activities because of the health problem, n (%) Capacity of the environment to respond to the patient’s requirement, n (%) Capacity to ask for needed health services (i.e. home care, transportation), n (%) Emotion and cognition Emotional reaction (sadness, anxiety, anger), n (%) Cognitive functioning (attention, orientation, judgment), n (%) Family network management Family caregivers concerns, availability of family members, n (%) Health services management Reasons for previous emergency department visit or hospitalisation, issues related to hospital, n (%) Knows why and when go back to the emergency department or the hospital, n (%) Knows which other external health care resources are available, n (%)

Gagnon et al., 1999; Spillane et al., 1997). Similar to the results in the present study, McCusker et al. (2003), Mion et al. (2003) and Hansagi et al. (2008) demonstrated significant improvements in secondary outcomes, but not on emergency department revisits. These three studies focused on either frequent users or elders, and were conducted in three different countries (Canada, United States and Sweden), suggesting that the clientele and health care systems were not influential in the success or not of the interventions tested. The observed impact of the current intervention on improved self-care capacities may be explained by the intervention’s direct focus on perceived concerns, evaluated with the clinical intervention scale. This scale, used by the project nurse at the beginning of each encounter, assessed the patient’s ability to cope with concerns related to self-care – for instance, illness and treatment, medication, and activities of daily living. However, targeting and possibly reducing some concerns about self-care issues did not reduce emergency department revisits. When the underlying illness-related problems were not resolved at the initial visit, patients may have been better able to deal with self-care for these problems, but it was not as likely that emergency department revisits for the ongoing illness problems could have been reduced by the program that we tested. Patients suffering from multiple and/or complex health problems may need more complex, multifactorial interventions. No impact of the intervention was observed

2 5 4 2

(0.9%) (3.7%) (0.9%) (5.6%)

11 (10.2%) 1 (0.9%)

6 (5.6%) 2 (1.9%)

12 (11.1%) 0

1 (0.9%)

4 (3.7%)

8 (7.4%)

4 (3.7%)

1 (0.9%)

3 (2.8%)

11 (10.2%) 3 (2.8%)

9 (8.3%) 3 (2.8%)

10 (9.3%) 2 (1.9%)

for medication adherence and there may be several explanations for this. While most patients reported no adherence problems on the one-month adherence scale, the scale assessed overall adherence rather than adherence for each medication separately, and self-report scales may overestimate the adherence rate. Additionally, a change in usual care may have reduced differences between the groups, since a full time pharmacist was assigned to the emergency department during the course of the study, thus giving all patients access to pharmacist interventions. The impact of the study intervention directed at helping patients with their medications may have been attenuated by this co-intervention. There was an observed impact of the intervention on three dimensions of continuity of care: informational, relational, and management. In the experimental group the project nurse used the clinical intervention tool at the beginning of each encounter to assess health care links, and was therefore able to resolve some concerns in this area by referring patients with difficulties to community resources outside the emergency department or to the emergency department. This may have contributed to higher scores on the continuity of care scale for patients in the experimental group. Continuity of care also impacts quality of care because it reflects how information is communicated between professionals, patients and family members. The intervention’s impact of improved patient’s perception of continuity of care in this area provides

S. Cossette et al. / International Journal of Nursing Studies 52 (2015) 666–676

674

Table 5 Effect of the intervention on secondary outcomes at 30 days. Experimental (n = 108)

Control (n = 95)

p value (95% CI)

Heart Continuity of Care Questionnaire Total score, mean (SD) Informational continuity subscale, mean (SD) Relational continuity subscale, mean (SD) Management continuity subscale, mean (SD)

133.77 (29.21) (n = 107) 55.15 (14.28) (n = 108) 28.79 (7.48) (n = 107) 18.66 (7.71) (n = 107)

119.73 (27.94) (n = 95) 50.12 (13.52) (n = 95) 26.57 (8.06) (n = 95) 15.89 (7.30) (n = 95)

.003a .011a .044a .010a

Illness Perception Questionnaire-Revised Consequences of the illness, mean (SD) Perceived personal control, mean (SD) Perceived treatment control, mean (SD)

19.25 (4.97) (n = 107) 22.17 (4.65) (n = 107) 20.14 (3.32) (n = 106)

19.50 (5.31) (n = 93) 21.29 (4.32) (n = 93) 19.15 (3.22) (n = 91)

.73a ( 1.69, 1.18) .17a (- 37, 2.14) .037a (.061, 1.91)

Therapeutic Self-Care Tool, mean (SD)

52.41 (6.75) (n = 101)

50.13 (6.79) (n = 91)

.021a (.34, 4.21)

Hospital Anxiety and Depression Scale-Anxiety Subscale (continuous score), mean (SD) Hospital Anxiety and Depression Scale-Deprive symptoms Subscale (continuous score), mean (SD)

4.43 (3.62) (n = 107)

5.95 (4.29) (n = 94)

.007a ( 2.62,

.42)

2.82 (3.15) (n = 107)

3.84 (3.93) (n = 94)

.043a ( 2.00,

.032)

Hospital Anxiety and Depression Scale-Anxiety clinical score, n (%) Normal: between 0 and 7 Borderline: between 8 and 10 Abnormal: between 11 and 21

.025b 85/107 (79.4%) 16/107 (15.0%) 6/107 (5.6%)

69/94 (73.4%) 9/94 (9.6%) 16/94 (17.0%) .47b

Hospital Anxiety and Depression Scale-Depressive symptoms clinical score, n (%) Normal: between 0 and 7 Borderline: between 8 and 10 Abnormal: between 11 and 21

96/107 (89.7%) 7/107 (6.5%) 4/107 (3.7%)

80/94 (85.1%) 7/94 (7.4%) 7/94 (7.4%)

Self-Reported Medication-Taking Scale, n (%) Never miss Rx 1 missing Rx

74/106 (69.8%) 32/106 (30.2%)

58/94 (61.7%) 36/94 (38.3%)

a b

(4.08, 19.99) (1.17, 8.90) (.060, 4.37) (.68, 4.86)

.23b

Analysis of variance. Chi-square test.

evidence that a nursing intervention specially designed to assess patient concerns before discharge can improve this recognized quality of care indicator (Schull et al., 2010). We observed some differences in symptoms of anxiety and depression, considered in Andersen’s model (1995) to be predisposing factors for emergency department revisits. Again, nurses may have been able to help patients in the experimental group with these concerns by providing reassurance or referrals to appropriate psychological support resources available at the emergency department.

8. Limitations The present study has limitations. The most significant threat to validity is the number of patients lost to followup in the present sample. These losses were more frequent in the control group than the experimental group, and more common in less educated subjects. The study sample was therefore more educated, and more willing to participate in the study, and thus possibly more receptive to the intervention. The loss of patients to follow-up, especially in the control group, attenuates the efficacy of randomization in a clinical trial. Therefore, a comparison of baseline characteristics was carried out which revealed four differences between the two groups based on pvalues < .10 or clinical importance: driving status, ambulance transportation to the hospital, mean number of medications and gender. After controlling for each of these

variables (one at a time) the secondary outcome results did not change. Another limitation of the study is the fact that the study hospital is dedicated to cardiac care. Although one-third of the study patients were consulting the emergency department for non-cardiac problems, this nevertheless limits generalizability of the results. Other limitations to generalizability are that the sample was restricted to French- or English-speaking patients, as well as patients discharged directly home from the emergency department. Finally, the sample size was based on the primary outcome of emergency department revisits, and no power analysis was performed for the secondary outcomes examined in the present paper. Strengths of the trial include a large sample size, allocation concealment, and blinding of the analyst. Additionally, randomization was supervised by an external coordinating center, and CONSORT guidelines were followed throughout the study. 9. Conclusion A major challenge in the randomized trial testing of nursing interventions is the choice of sensitive patient outcomes. Therefore, improvement in nursing-sensitive outcomes such as self-care capacities and continuity of care is important in the overall context of improving quality of life for patients consulting the emergency department. These secondary outcomes have been

S. Cossette et al. / International Journal of Nursing Studies 52 (2015) 666–676

suggested as potentially important influences on emergency department revisits. The current intervention’s impact on these secondary outcomes without resulting in a change in emergency department revisit rates suggests that some other unmeasured factors may be more important in influencing emergency department revisits. Future research is needed to better understand the complex relationship between emergency department utilization and a variety of intermediary factors in order to develop interventions that will optimize emergency department utilization. Conflict of interest: The authors report no conflicts of interest. Funding: Funding was obtained from the Fonds de la Recherche Que´bec Sante´ (FRQS) (Grant No. 10187), the Quebec Network on Nursing Intervention Research (RRISIQ), and the Montreal Heart Institute Foundation and Research Center. Ethical approval: Montreal Heart Institute Research Ethics Board, 2007-52, 963. Acknowledgments We thank the patients who generously agreed to participate. We thank the former Emergency Department Chief Nurse Danielle Perreault, RN, BSc for her contribution to the study supervision, the Director of Nursing who supported the project (Marie-He´le`ne Carbonneau, RN, MSc) and the project nurses including Francois Pilon, RN, BSc., Ve´ronique Paradis, RN, MSc, Johanne Vezina, RN, BSc and Johanne Loyer, RN, BSc. We thank the members of the interim analysis committee, Drs. Marc Afilalo, Danielle Roberge, and Jacques Lelorier. Writing and editing consultation was provided by Kate Johnson, BA (Joint Hons), BJ (Hons). References Althaus, F., Paroz, S., Hugli, O., Ghali, W.A., Daeppen, J.B., PeytremannBridevaux, I., Bodenmann, P., 2011. Effectiveness of interventions targeting frequent users of emergency departments: a systematic review. Ann. Emerg. Med. 58 (1) 41–52, http://dx.doi.org/10.1016/ j.annemergmed.2011.03.007, e42. Andersen, R.M., 1995. Revisiting the behavioral model and access to medical care: does it matter? J. Health Soc. Behav. 36 (1) 1–10. http://www.ncbi.nlm.nih.gov/pubmed/7738325 (accessed 15.06.14). Ashton, C.M., Wray, N.P., 1996. A conceptual framework for the study of early readmission as an indicator of quality of care. Soc. Sci. Med. 43 (11) 1533–1541. http://www.ncbi.nlm.nih.gov/pubmed/8961397 (accessed 15.06.14). Bullard, M.J., Unger, B., Spence, J., Grafstein, E., 2008. Revisions to the Canadian Emergency Department Triage and Acuity Scale (CTAS) adult guidelines. Can. J. Emerg. Med. 10 (2) 136–151. http://www. ncbi.nlm.nih.gov/pubmed/18371252 (accessed 15.06.14). Canadian Institute of Health Information (CIHI), 2005. Understanding Emergency Department Wait Times. Who is Using Emergency Department and How Long Are They Waiting? CIHI, Ottawa, ON, Canada. https://secure.cihi.ca/free_products/Wait_times_e.pdf (accessed 15.06.14). Caplan, G.A., Williams, A.J., Daly, B., Abraham, K., 2004. A randomized, controlled trial of comprehensive geriatric assessment and multidisciplinary intervention after discharge of elderly from the emergency department – the DEED II study. J. Am. Geriatr. Soc. 52 (9) 1417–1423, http://dx.doi.org/10.1111/j. 1532-5415.2004.52401.x. Cossette, S., Frasure-Smith, N., Lespe´rance, F., 2001. Clinical implications of a reduction in psychological distress on cardiac prognosis in patients participating in a psychosocial intervention program.

675

Psychosom. Med. 63 (2) 257–266. http://www.ncbi.nlm.nih.gov/ pubmed/11292273 (accessed 15.06.14). Cossette, S., Frasure-Smith, N., Lespe´rance, F., 2002. Nursing approaches to reducing psychological distress in men and women recovering from myocardial infarction. Int. J. Nurs. Stud. 39 (5) 479–494. http:// www.ncbi.nlm.nih.gov/pubmed/11996869 (accessed 15.06.14). Cossette, S., Vadeboncoeur, A., Frasure-Smith, N., McCusker, J., Guertin, M.-C., 2013. Randomized controlled trial of a nursing intervention to reduce emergency department revisits. Can. J. Emerg. Med. 15 (9) 1– 8, http://dx.doi.org/10.2310/8000.2013.131291. Current Controlled Trials, 2008. http://www.controlled-trials.com/. Doran, D., Harrison, M.B., Laschinger, H., Hirdes, J., Rukholm, E., Sidani, S., Hall, L.M., Tourangeau, A.E., Cranley, L., 2006. Relationship between nursing interventions and outcome achievement in acute care settings. Res. Nurs. Health 29 (1) 61–70, http://dx.doi.org/10.1002/ nur.20110. Fealy, G., McCarron, M., O’Neill, D., McCallion, P., Clarke, M., Small, V., O’Driscoll, A., Cullen, A., 2009. Effectiveness of gerontologically informed nursing assessment and referral interventions for older persons attending the emergency department: Systematic review. J. Adv. Nurs. 65 (5) 934–935, http://dx.doi.org/10.1111/j.13652648.2009.04961.x. Frasure-Smith, N., Lespe´rance, F., Prince, R.H., Verrier, P., Garber, R.A., Juneau, M., Wolfson, C., Bourassa, M.G., 1997. Randomised trial of home-based psychosocial nursing intervention for patients recovering from myocardial infarction. Lancet 350 (9076) 473–479, http:// dx.doi.org/10.1016/S0140-6736(97)02142-9. Gagnon, A., Schein, C., McVey, L.N., Bergman, H., 1999. Randomized controlled trial of nurse case management of frail older people. J. Am. Geriatr. Soc. 47 (9) 1118–1124. http://www.ncbi.nlm.nih.gov/ pubmed/10484257 (accessed 15.06.14). Hadjistavropoulos, H.D., Biem, H.J., Kowalyk, K.M., 2004. Measurement of continuity of care in cardiac patients: reliability and validity of an inperson questionnaire. Can. J. Cardiol. 20 (9) 883–891. http://www. ncbi.nlm.nih.gov/pubmed/15266358 (accessed 15.06.14). Hansagi, H., Olsson, M., Hussain, A., Ohlen, G., 2008. Is information sharing between the emergency department and primary care useful to the care of frequent emergency department users? Eur. J. Emerg. Med. 15 (1) 34–39, http://dx.doi.org/10.1097/MEJ.0b013e3282aa4115. Lang, E., Afilalo, M., Vandal, A.C., Boivin, J.F., Xue, X., Colacone, A., L?ger, R., Shrier, I., Rosenthal, S., 2006. Impact of an electronic link between the emergency department and family physicians: a randomized controlled trial. Can. Med. Assoc. J. 174 (3) 313–318. http://www.cmaj. ca/content/174/3/313.full.pdf+html (accessed 15.06.14). McCusker, J., Jacobs, P., Dendukuri, N., Latimer, E., Tousignant, P., Verdon, J., 2003. Cost-effectiveness of a brief two-stage emergency department intervention for high-risk elders: results of a quasi-randomized controlled trial. Ann. Emerg. Med. 41 (1) 45–56, http://dx.doi.org/ 10.1067/mem.2003.4. McCusker, J., Verdon, J., 2006. Do geriatric interventions reduce emergency department visits? A systematic review. J. Gerontol. Ser. A: Biol. Sci. Med. Sci. 61 (1) 53–62. http://biomedgerontology. oxfordjournals.org/content/61/1/53.long (accessed 15.06.14). Mion, L.C., Palmer, R.M., Meldon, S.W., 2003. Case finding and referral model for emergency department elders: a randomized clnical trial. Ann. Emerg. Med. 41, 57–68., In: http://www.ncbi.nlm.nih.gov/ pubmed/?term=12514683. Coordinating Center. Available at: http://mhicc.org. Morisky, D.E., Green, L.W., Levine, D.M., 1986. Concurrent and predictive validity of a self-reported measure of medication adherence. Med. Care 24, 67–74. http://www.ncbi.nlm.nih.gov/pubmed/ ?term=3945130 (accessed 15.06.14). Mosca, L., Benjamin, E.J., Berra, K., Bezanson, J.L., Dolor, R.J., Lloyd-Jones, D.M., Wenger, N.K., 2011. Effectiveness-based guidelines for the prevention of cardiovascular disease in women –2011 update: a guideline from the American Heart Association. J. Am. Coll. Cardiol. 57 (12) 1404–1423. http://dx.doi.org/10.1016/j.jacc.2011.02.005 (accessed 15.06.14). Schoen, C., Osborn, R., Huynh, P.T., Doty, M., Davis, K., Zapert, K., Peugh, J., 2004. Primary care and health system performance: adults’ experiences in five countries. Health Affairs Suppl Web Exclusives, http:// dx.doi.org/10.1377/hlthaff.w4.487, W4-487-503. Schull, M.J., Hatcher, C.M., Guttmann, A., Leaver, C.A., Vermeulen, M., Rowe, B.W., Anderson, G.M., Zwarenstein, M., 2010. Development of a consensus on evidence-based quality of care indicators for canadian emergency departments. ICES investigative report, Toronto, ON, Canada, In: http://caep.ca/sites/default/files/caep/files/ national_ed_quality_indicators-10mar2010.pdf accessed 15/06/2014. Shumway, M., Boccellari, A., O’Brien, K., Okin, R.L., 2008. Cost-effectiveness of clinical case management for ED frequent users: results of a

676

S. Cossette et al. / International Journal of Nursing Studies 52 (2015) 666–676

randomized trial. Am. J. Emerg. Med. 26 (2) 155–164, http:// dx.doi.org/10.1016/j.ajem.2007.04.021. Sidani, S., 2003. In: Doran, D.M. (Ed.), Self-Care. Jones and Bartlett, Sudbury, MA. Snapinn, S., Chen, M.G., Jiang, Q., Koutsoukos, T., 2006. Assessment of futility in clinical trials. Pharm. Stat 5 (4) 273–281, http://dx.doi.org/ 10.1002/pst.216. Spillane, L.L., Lumb, E.W., Cobaugh, D.J., Wilcox, S.R., Clark, J.S., Schneider, S.M., 1997. Frequent users of the emergency department: can we intervene? Acad. Emerg. Med. 4 (6) 574–580, http://dx.doi.org/ 10.1111/j. 1553-2712.1997.tb03581.x. Twisk, J., de Vente, W., 2002. Attrition in longitudinal studies. How to deal with missing data. J. Clin. Epidemiol. 55 (4) , http://dx.doi.org/ 10.1016/S0895-4356(01)00476-0, 329–227. Vadeboncoeur, A., Kennedy, W., Bourdages, T., Vaillant, L., Perreault, D., Poirier, D., Laurier, C., 2003. Risk factor predicting return to the

emergency department by patients using the emergency department of the Montreal Heart Institute. In: The Second Mediterranean Emergency Medicine Congress. Barcelona, Spain, (accessed 15.06.14), p. 156. Weinman, J., Petrie, K.J., Moss-Morris, R., Horne, R., 1996. The illness perception questionnaire: a new method for assessing the cognitive representation of illness. Psychol. Health 11, 431–445. http://www. tandfonline.com/doi/abs/10.1080/08870449608400270#. UpeVnlOfZI0 (accessed 14.06.14). Zigmond, A.S., Snaith, R.P., 1983. The Hospital Anxiety and Depression Scale. Acta Psychiatr. Scand. 67, 361–370. http://www.ncbi.nlm.nih. gov/pubmed/6880820 (accessed 14.06.14). Zwarenstein, M., Treweek, S., Gagnier, J.J., Altman, D.G., Tunis, S., Haynes, B., Oxman, A.D., Moher, D., 2008. Improving the reporting of pragmatic trials: an extension of the CONSORT statement. Br. Med. J. 337 (a2390) 1–8, http://dx.doi.org/10.1136/bmj.a2390.

The impact of an emergency department nursing intervention on continuity of care, self-care capacities and psychological symptoms: secondary outcomes of a randomized controlled trial.

As yet there is no firm evidence about the types of intervention that can reduce emergency room revisits. However, the literature on emergency room re...
394KB Sizes 1 Downloads 7 Views

Recommend Documents