Research Article For reprint orders, please contact: [email protected]

Engaging stakeholders to design a comparative effectiveness trial in children with uncontrolled asthma

Aim: To present the methods and outcomes of stakeholder engagement in the development of interventions for children presenting to the emergency department (ED) for uncontrolled asthma. Methods: We engaged stakeholders (caregivers, physicians, nurses, administrators) from six EDs in a three-phase process to: define design requirements; prototype and refine; and evaluate. Results: Interviews among 28 stakeholders yielded themes regarding in-home asthma management practices and ED discharge experiences. Quantitative and qualitative evaluation showed strong preference for the new discharge tool over current tools. Conclusion: Engaging endusers in contextual inquiry resulted in CAPE (CHICAGO Action Plan after ED discharge), a new stakeholder-balanced discharge tool, which is being tested in a multicenter comparative effectiveness trial. First draft submitted: 14 July 2015; Accepted for publication: 4 October 2015; Published online: 21 December 2015 Keywords:  asthma • health communication • patient discharge • pediatrics • stakeholder engagement • written action plan

Comparative effectiveness research is intended to address the expressed needs of patients, caregivers and other decision-makers in healthcare [1] . Such engagement is considered critical to evaluating interventions relevant to endusers and that are feasible for use in real world clinical settings. National and international asthma guidelines recommend the use of written instructions to promote appropriate use of medications, avoidance of environmental triggers and advice about when to seek additional medical attention [2,3] . However, systematic reviews indicate that the content, format and benefits of such written instructions (usually called action plans) are highly variable, with relatively low rates of use because most action plans are designed by teams of medical experts with relatively little input from patients, ­caregivers and clinicians [4,5] . The CHICAGO study is a multicenter comparative effectiveness trial funded by the Patient-Centered Outcomes Research Insti-

10.2217/cer.15.52 © 2016 Future Medicine Ltd

tute (PCORI) to test strategies to improve the care and outcomes of African–American and Latino children with uncontrolled asthma presenting to the emergency department (ED) in Chicago [6] . In Chicago, African–American and Latino children aged 5–11 years bear a disproportionate share of the burden from asthma [7] . Among the most visible of these disparities is the five- to seven-fold higher rate of visits to the ED for uncontrolled asthma in communities with a high proportion of African–American and Latino children compared with other communities  [Department of Public Health, Chicago, Pers. Comm.]  . As part of the PCORI-funded CHICAGO study, we employed user-centered design methods to engage caregivers, clinicians and administrators in the development and evaluation of an asthma discharge tool for African–American and Latino children who present to the ED for uncontrolled asthma [8] . To our knowledge, this is the first

J. Comp. Eff. Res. (2016) 5(1), 17–30

Kim Erwin*,1, Molly A Martin2, Tara Flippin1, Sarah Norell1, Ariana Shadlyn1, Jie Yang1, Paula Falco1, Jaime Rivera1, Stacy Ignoffo3, Rajesh Kumar4, Helen Margellos-Anast5, Michael McDermott6, Kate McMahon7, Giselle Mosnaim8, Sharmilee M Nyenhuis2, Valerie G Press9, Jessica E Ramsay5, Kenneth Soyemi10, Trevonne M Thompson2 & Jerry A Krishnan2,11 IIT Institute of Design, 350 N LaSalle, Chicago, IL 60654, USA 2 University of Illinois at Chicago, 1200 W Harrison St Chicago, IL 60607, USA 3 Chicago Asthma Consortium, PO Box 31757, Chicago, IL 60631, USA 4 Ann & Robert H Lurie Children’s Hospital of Chicago, 225 E Chicago Ave., Chicago, IL 60611, USA 5 Sinai Health System, California Avenue, 15th Street, Chicago, IL 60608, USA 6 Illinois Emergency Department Asthma Surveillance Project (IEDASP) 7 Respiratory Health Association, 1440 W Washington Blvd, Chicago, IL 60607, USA 8 Rush University Medical Center, 1653 W Congress Pkwy, Chicago, IL 60612, USA 9 University of Chicago, 5801 S Ellis Ave., Chicago, IL 60637, USA 10 John H Stroger Jr Hospital of Cook County, 1901 W Harrison St Chicago, IL 60612, USA 11 University of Illinois Hospital & Health Sciences System, 1740 W Taylor St Chicago, IL 60612, USA *Author for correspondence: kerwin@ id.iit.edu 1

part of

ISSN 2042-6305

17

Research Article  Erwin, Martin, Flippin et al. report of methods and outcomes of end-user engagement to develop an asthma discharge tool tailored to high-risk children for a comparative effectiveness trial. We are now testing the effectiveness of the asthma discharge tool on implementation and clinical outcomes in the multicenter CHICAGO pragmatic clinical trial (ClinicalTrials.gov NCT02319967) [9] . Methods We engaged four stakeholder groups: caregivers of minority children with a history of uncontrolled asthma presenting to the ED in the past 12 months; ED physicians and nurses; ED administrators; and outpatient clinicians. Stakeholders from six EDs in Chicago that would serve as the centers for the clinical trial (CHICAGO ED clinical centers) were engaged in a three-phase process that employed various methods of contextual inquiry (Figure 1) : define design requirements with multistakeholder user input; prototype and refine to shape a discharge tool that

Methods

fits the content and support needs of all key stakeholders; and evaluate end-user stakeholder preferences for the new and existing discharge tools. The study was approved by institutional review boards (IRBs) at all participating institutions, IRB #20140412, 2014-056, 2014-15829, MSH #14-10, 14-0534, 14-095 and 13083001-IRB01 at the University of Illinois at Chicago (IL, USA), Illinois Institute of Technology (IL, USA), Lurie Children’s Hospital (IL, USA), Mount Sinai Hospital (IL, USA), University of Chicago Medicine (IL, USA), Cook County Hospital (IL, USA) and Rush University Medical Center (IL, USA), respectively. A convenience sample of physician and nonphysician clinicians and administrators from each of the participating six ED clinical centers participated as key informants. Key informants were selected to be representative of their clinical center in their ability to speak to the processes and materials involved in discharge from the ED and as active end-users on the clinical side.

Phase 1

Phase 2

Phase 3

Define requirements

Prototype + refine

Evaluate

Contextual stakeholder interviews Use of prototypes and toolkits with stakeholders to draw out design requirements

Iterative prototyping and testing using stakeholder reviews

Gibson survey to assess purpose, appearance, content, usefulness and overall impact Card sort activity to elicit qualitative responses

20 total

Patient caregivers

B B B B B B D D E

B B E

B B B

ED doctors

A

A B E

A A A B B B

15 total

ED nurses

A B C E

E

A A A B B B

11 total

ED admins

A B C D

C D

Outpatient providers

A B C D F

Results

B C D E F

B B B B

6 total 5 total

28 total stakeholders

9 total stakeholders

20 total stakeholders

Discharge tool requirements

Final discharge tool Illustration-driven format and medication-optimized instruction

Strong multi-stakeholder preference for the final discharge tool

Vocabulary clarity Page count and focus

57 total

Visual presentation and format Medication instruction Conversation support

A

= participant’s recruitment site; all participants recruited from one of six participating Chicago-based EDs

Figure 1. We employed a three-phase design process to develop a stakeholder-balanced asthma discharge tool. Stakeholders included patient caregivers, emergency department clinicians (physicians, nurses, administrators) and ambulatory (outpatient) physicians. A highlight of findings from each stage is also presented. ED: Emergency department.

18

J. Comp. Eff. Res. (2016) 5(1)

future science group

Engaging stakeholders to design a comparative effectiveness trial in children with uncontrolled asthma 

Research Article

Figure 2. Sample projectives used with stakeholders in Phase I (from left to right): a caregiver-drawn asthma journey map; a physician-annotated prototype discharge tool; three sample discharge tools built by outpatient providers from a toolkit of preprinted sticky notes.

Similarly, a convenience sample of caregivers of black or Hispanic/Latino children who had visited at least one of the six ED clinical centers in the past 12 months participated in focus groups, user-centered home observations or both. Previous studies have established the critical role of triggers in the home environment and inadequate asthma controller use as risk factors for asthma exacerbations and emergency department visits in children with asthma. Thus, the asthma discharge tool included a focus on both environmental control and appropriate use of asthma medications [10–14] . Phase I: define design requirements

Data collection in Phase I employed a user-centered design approach that combines field interviews conducted onsite in a user’s home or workplace with direct observation of users engaged in relevant tasks. These methods come from the field of design, which are not typically used in qualitative health research. The primary difference is that design employs a contextual approach that focuses on context of use – not just a tool or its content – and conducts the inquiry using user-centered observations. [15] . Central to contextual inquiry is the belief that human actions can only properly be understood in context, and that all activity is informed by immediate circumstances and therefore is best observed ‘in the wild’ rather than in a lab or through recounting past events with others in focus groups  [16] . We, therefore, conducted all inquiry in clinical settings in the ED and ambulatory sites, and in the homes of caregivers. We also employed projectives, which in this study were prototype discharge documents, toolkits from which physicians could construct their own discharge document and probes consisting of stakeholder-drawn visualizations of relevant past and present experiences of asthma management and ED experiences (Figure 2) . We conducted 19 key informant stakeholder interviews with physicians, nurses and administrators from the ED,

future science group

and outpatient clinicians. All interviews were conducted in the ED or ambulatory setting for 60–80 min. Two interviewers were present, and interviews were audiorecorded and photographed. The key informant protocol covered three domains: ED patient discharge experience using open-ended questions, discharge simulations and role play to target challenges and barriers in preparing a patient/caregiver for discharge; asthma treatment recommendations on ED discharge, including barriers to implementing these components in their ED or practice; and prototype discharge tool review to elicit feedback about preferred discharge instructions. We also conducted interviews with caregivers of African–American and Latino children with a history of an ED visit for uncontrolled asthma in the past 12 months in one or more of the CHICAGO ED clinical centers. Eight of the caregivers were African–American and one was Latino, seven were single mothers, seven had other children and seven had other family members diagnosed with asthma. All interviews were conducted in the home and ranged from 2 to 3 h. They were audio-recorded and transcribed. A digital camera was used to document the home environment and location of relevant artifacts, such as storage of medications and discharge documents. Interviews targeted baseline asthma knowledge, self-management practices and recent ED discharge experience across four touchpoints – waiting, triage, treatment and discharge. The approach consisted of open-ended questions, review of a prototype discharge document and use of multiple probes and activities to help caregivers express asthma care experiences. All interview data were coded and analyzed using principles of Grounded Theory to identify recurring patterns of beliefs, interactions, behaviors and needs across stakeholder groups and clustered into design requirements and opportunity areas for concept development  [17,18] . Analysts worked in pairs to ensure intercoder reliability (S Norell, J Rivera and T Flippin).

www.futuremedicine.com

19

Research Article  Erwin, Martin, Flippin et al. Phase II: prototype & refine

A total of 9 caregivers, ED clinicians (physicians and nurses) and ED administrators were engaged in two iterations of assessment and refinement to converge on a single discharge tool that incorporated health literacy and information design principles (maximum Flesch–Kincaid 6th reading grade level; reduced word count, sentence length, text blocks and medical jargon; consistent use of typographic hierarchy and underlying grid; and key information presented in illustration and callouts) [19–24] . We also evaluated the Flesch–Kincaid reading level of existing discharge documents at the CHICAGO ED clinical centers. The CHICAGO investigators (clinicians, social scientists, community health workers and supervisors) then collaborated in providing feedback to the design team to finalize an asthma discharge tool for use in children presenting to the ED with uncontrolled asthma. Phase III: evaluate stakeholder preferences

We then assessed preferences among caregivers and ED clinicians by comparing the documents currently in use in two different CHICAGO ED clinical centers with the newly developed tool. We employed a published quantitative assessment tool (Gibson survey) that evaluates five domains (purpose, appearance, usefulness, overall impact and for clinicians, content) developed for patients/caregivers (15 items) and clinicians (30 items)  [25] . Participants were asked to respond using a five-point Likert scale (strongly disagree to strongly agree); possible scores are 1–5, with higher scores indicating greater preference. To understand the rationale for preferences among caregivers and clinicians, we also employed qualitative methods using a card sorting activity. Card sorting works to surface mental models and evaluate participant agreement by providing a stack of cards containing phrases or words and asking each participant to sort the cards into categories, as makes sense to them [26,27] . A total of 11 cards were presented to clinicians, with another 11 to caregivers. All cards were crafted with first-person statements, such as “I think this document provides more guidance for my patients after discharge” and were structured to capture both participant behaviors and attitudes. Responses were tabulated and assessed for patterns across and within stakeholder segments using data visualization software Nineteen [28] . Results Phase I: define design requirements

We conducted 28 on-site stakeholder interviews with nine caregivers, six ED physicians, four ED nurses, four ED administrators and five outpatient clinicians (Figure 1) . Analysis of contextual research data produced eight themes for in-home asthma management

20

J. Comp. Eff. Res. (2016) 5(1)

practices and three themes related to ED discharge experiences (Table 1) . Tools to share information, coordinate care

Caregivers expressed the need for a discharge tool that facilitates communication, education and coordination of care with others (babysitters, extended family, school, daycare and camp personnel) who share in the care of their children. Discharge information stored out of sight

Current discharge documents are often stored in bags, drawers and with stacks of bills, so are not easily available for reference if needed. No caregivers had asthma-related information on display in the home. Learning through trial & error causes gaps in understanding even in experienced caregivers

The sheer number of contributing factors can make the logic of asthma exacerbations hard to piece together. While our caregivers reported prioritizing their children’s health – by moving, leaving their jobs, delaying promotions to be more available to care for their children, vigorous cleaning or keeping kids inside – interviews suggested their management strategies are often based on an incomplete understanding of asthma. The resulting failure of proactive efforts can promote reactive behavior (i.e., ED visits). Medication workarounds

Medication adherence is tough even for informed caregivers. Inhalers run out, causing families to share or to substitute reliever medicine. Because childcare usually occurs outside the home, shared inhalers must travel to multiple sites and return home without getting lost. Medication confusion

Many caregivers and outpatient clinicians reported confusion between reliever and controller inhalers. Caregivers also expressed distrust of continuous steroid use and potential side effects on their growing child, affecting their commitment to prescribed regimens. Patient education positioned at the weakest moment

Discharge protocols position patient education at the moment when patient caregivers are least prepared to take advantage of it – in the last 20 min of a typically 3–6 h (or longer) ED visit. Several nurses expressed frustration with caregivers’ unwillingness or inability to focus on patient education at discharge. Caregivers also reported frustration related to lengthy discharge processes, repetition of information they say

future science group

future science group

Mental model of asthma pieced together through trial and error, personal experience Logic of asthma hard to see, plan for Well-intended actions often ineffective

Inhalers do not last as long as they should Families share inhalers Substitute with ‘rescue’ medicine inhalers need to travel

3. Learning through trial and error causes gaps in understanding in even experienced caregivers  

4. Medication workarounds  

www.futuremedicine.com

ED: Emergency department.

“Parents do not want to sit here and listen to this stuff when their ride is waiting outside and they have already been here for 4–6 h, and they are tired and hungry and miserable and their kid is screaming”– ED physician “Upon discharge, they give you the whole work-up – discharge instructions, a number to call for emergencies, follow-up appointments, information on asthma. They do it every time. Even if we are so used to the information we do not need it any more” – Caregiver “4 h is the most we have stayed there. I am just ready to go, because of the other three (kids) I have at home” – Caregiver

6. Patient education positioned at the weakest moment in discharge  

Caregiver fatigue Repetitive information Neglecting other family

“Fast acting or rescue medication? Honestly? I never knew what the controller medicine was to what is a rescue medicine” – Caregiver “This Proventil sounds like preventative. So I might have said take Flovent, the orange pump… and the patient comes back with a yellow and orange Proventil pump and says yeah, this is the prevent one. I take two puffs every morning and two puffs every night. I am like no! I can not even tell you how many times that has happened” – Outpatient provider “When they first sent him home with the medication I was like I am not giving him all that medication all the time, he does not need it” – Caregiver

5. Medication confusion   Inhaler mix-ups Medication names sound alike ‘Better = cured’

“If my son uses up the inhaler within 2 or 3 weeks, what am I supposed to do for the other week? Because you have 4 weeks in the month” – Caregiver “My mom is an asthmatic, so we will go to her house and get some asthma spray (when ours runs out)” – Caregiver “If we are out, I have no choice but to use the rescue inhaler” – Caregiver “I have visits where …they are bringing the babysitter and the grandma and the dad and the mom. And they are trying to figure out how they are going to transfer the kids’ medications between all these sites”– Outpatient provider

“I hope she does not catch (an asthma attack) today because it is raining at one point, then it is hot at another point” – Caregiver “Every time he has had an asthma attack, it has been different situations. So we do not really know what his trigger is. The initial time it was fresh cut grass that set him off. It has been a change of temperature, it has been if you spray something, if he has been around somebody with a cold. And so it is very hard to say, okay, what triggered it this time?” – Caregiver “Germs are a trigger … so I clean. I bleach. I am a bleach fanatic” – Caregiver

“I keep them in a bin where I keep all my mail and stuff. Then after a year or two, I go through it, and what ever needs to be thrown out, I throw out” – Caregiver “It is pretty much duplicates of what we have. So I have it piled up. I have a ton of stuff to shred now” – Caregiver

Need for simplified instruction Stored, not displayed Repetitive information ignored

2. Discharge information stored out of sight  

Quotes “(You need to) not only educate yourself, but to educate others that you will leave your child with for long periods – babysitters, caregivers, teachers” – Caregiver “Maybe (we could have) a website to get this action care plan to print out and give to the school or caretaker? Something that you can educate somebody with in 2 min?” – Caregiver

Details

1. Tools to share Need to educate others information, coordinate Need to share information care  

Themes

Table 1. Themes from contextual research in the home and emergency department sites. Engaging stakeholders to design a comparative effectiveness trial in children with uncontrolled asthma 

Research Article

21

22

J. Comp. Eff. Res. (2016) 5(1)

they already know, and the need to manage work, other children and household obligations reduces receptivity to education at discharge.

“Most of the time I do not pay very much attention to the discharge paper because it offers me nothing. I can not imagine that it offers much to the patient” – ED nurse ”It gets to be 15 pages…you hand this to them and they are like, ‘What is all this stuff?’ If there is something important, I try to circle it” – ED physician ”We do not have good education tools, and everything falls back to the nurse at the end of the line” – ED nurse ”The value in a discharge document…is the teaching that happens when you are giving the document. The document should be a teaching tool, not only ‘the thing you receive” – Outpatient provider

Fragmented discharge experience

Operational realities can produce an ED discharge experience that is executed piecemeal and by multiple staff members. For example, sometimes the attending physician delivers medication instruction, a resident attempts patient education and a nurse prints and delivers paperwork. This workflow may change based on the number of patients waiting to be seen or acuity of illness among patients in the ED – the pressure to ‘treat and street’ was noted as a driver of practice variation. Clinical staff reported that continuity and predictability in discharge is difficult to provide using hand-offs, especially in EDs where poor site lines can limit communication between staff. Clinician/caregiver conversations not well-supported by existing tools

Caregivers often receive large amounts of information at an ED visit. Clinic staff across sites reported numerous challenges in communicating that information to caregivers in the ED. Many staff said they do not review discharge documents with patients, as time is tight and documents are long, complicated and hard to use with caregivers. Stakeholder input regarding the design of the discharge tool identified several design priorities for the discharge tool (Table 2) . Vocabulary clarity + reading level

Reduce complexity and simplify language to include children, more caregivers and others outside the family. Page count & focus

Remove extraneous content and streamline to focus on caregiver action steps. Visual presentation & format

Use illustrations and layout to make key ideas accessible and inclusive, while retaining a professional and serious appearance. Medication instruction ED: Emergency department.

8. Clinician, caregiver Tools are dense, text-heavy, hard conversations not to use well-supported by tools Conversation is the educational touchpoint

“Normally, the nurse will come back…with the prescriptions and further instructions. First the doctor always comes in and says whatever, then the nurse is the one who comes back and gives you your discharge” – Caregiver ”Multiple professionals could be responsible for (discharge), you know, so when multiple people could be involved, if each of those people is mediocre, you could get discharged with paperwork that says nothing” – ED nurse  7. Fragmented discharge ‘Treat and street’ Discharge distributed across staff process   Staff hand-off allows information to slip through the cracks

Quotes Details Themes

Table 1. Themes from contextual research in the home and emergency department sites (cont.).

Research Article  Erwin, Martin, Flippin et al.

Organize, clarify and detail all medication-related instruction so as to create a plan that caregivers can understand and follow. Conversation support

Simplify and structure the protocol for clinicians, support discussion of sensitive topics and make a tool caregivers can use with others.

future science group

future science group

Simplify language and sentence structure Use layman’s terms, not medical jargon

Reduce word count Promote action steps Remove extraneous material

Engage kids in care Earn a spot on the refrigerator Use visuals so as to include everyone Make easy to copy and share

Clearly organize medication types Clarify timing, duration and dosages Reinforce medication adherence as priority

1. Vocabulary clarity + reading level  

2. Page count + focus  

3. Visual presentation + format   

4. Medication instruction  

www.futuremedicine.com

ED: Emergency department.  

5. Conversation support    Standardize key messages Simplify discharge conversation Provide a shared tool to anchor the conversation Help caregivers participate Help caregivers share with others in their care circle

Details

Requirements

Table 2. Design requirements.

“Give me the simplest protocol I can remember while running from room to room” – ED physician ”I think the nurses themselves do not understand what they need to communicate to the patient, thinking that they (patients) would just know” – ED admin nurse  “I would say most of the time I talk a lot and when I am done I always ask if they have got questions and I do not get much” – ED nurse ”(If they bring this document) I would know that at least the education has started. It would help me here, too, because sometimes many show up without any paper work, and I have got 15 min to figure out what happened”– Outpatient provider

“The means of administering medications, how to use them, when to use them is very confusing. All the medications have two different names. Many of them come in multiple different forms. They may be given two different asthma pumps, and I think they do not know the difference or they may be asked to use equipment that they have no familiarity with, like a spacer or a nebulizer machine, or for that matter even be exposed to medication forms that they are not familiar with” – Outpatient provider  

“I need to explain to him about asthma and the tightening of the chest and all of that, because he needs to know … so that one day if I am not there, he can say it himself” – Caregiver ”It has one thing to educate caregivers, which is important, but in the end, it has got to be the child. The child is going to know when he cannot breathe” – ED nurse ”I would definitely put something (visual) on the front page so that the kid knows ‘it is mine” – ED admin nurse ”All patients have different learning styles, right? You might not be able to assess that fully in an emergency encounter. But you could hit on multiple areas. For example, use written education from the asthma plan and visual cues that patients can identify” – Outpatient provider

“There are so many pages that nobody looks at any of them…” – ED physician ”The paperwork that EDs give now comes from the EMR, so it is filled with great nuggets but there is totally unnecessary blather. When patients look at it, they cannot find the things they actually need” – Outpatient provider ”They give you all these sheets, and to be quite honest, as a parent, just tell me what is going on. Tell me what I need to do. I am not going to take time – although my son is sick, do not get me wrong – but I am not taking time to read through all your literature. Just get to the plan of action here” – Caregiver

“A lot of our patients cannot read, and their parents cannot read, and then you hand them an asthma action plan and do the best that you can” – ED doctor ”In the beginning, you are not familiar with the medical terms, so you kind of have to … explain it just a little simpler” – Caregiver ”I see things that confuse my patients. For example, many do not know what an MDI is, so call it something that they are going to understand”– ED nurse

Quotes

Engaging stakeholders to design a comparative effectiveness trial in children with uncontrolled asthma 

Research Article

23

Research Article  Erwin, Martin, Flippin et al.

An early-stage prototype...

...informs three new prototypes

...refined into a final discharge tool

Drafted collaboratively by CHICAGO Plan investigators;

1. Digital resource focused

A stakeholder balanced solution

Organizes content into four clear action steps: 1. Take medication 2. Follow-up with provider 3. Recognize symptoms 4. Stay on top of asthma (self-management instruction)

Includes QR codes that link to online asthma content to better accommodate differing levels of caregiver expertise 2. Medication optimized Dedicates an entire page to clarifying inhaler types and medication regimen 3. Visual learning focused Uses illustration to represent key vocabulary and selfmanagement concepts to address low-literacy populations and children

Structured around four action steps Illustration-driven format from prototype 3 with supporting text at a reading grade level of 4.7 (Flesch–Kincaid) Also integrates stakeholder preferred medication instruction from prototype 2 and a digital link from prototype 1

Figure 3. In Phase II, an early prototype was refined through iterative stakeholder input to develop a new discharge tool, CAPE (CHICAGO Action Plan after emergency department discharge). CHICAGO: Coordinated Healthcare Interventions for Childhood Asthma Gaps in Outcome; QR: Quick response.

Phase II: prototype & refine

Nine stakeholders participated in prototype reviews (three caregivers; three ED physicians; one ED nurse and two ED administrators, Figure 1). In these reviews, stakeholders were presented with three potential discharge tool concepts based on Phase I results and were asked to engage in two cycles of review and collaborative editing (Figures 2 & 3) . Across stakeholder groups, highly-illustrated visual learning concepts earned the strongest favorable responses for its simplicity, clarity, visual appeal and ease of use. Literacy levels of patients and caregivers were a recurring topic in the formative research. Many ED physicians asked for solutions that used visual strategies to offset the complexity of text. ED nurses sought tools that could engage children directly in self-management. Caregivers also expressed confusion and impatience with materials that were hard to read, apply and share. In response, we customized illustrations to explain key vocabulary and selfmanagement concepts and employed the traffic light construct of green/yellow/red zones to aid caregiver assessment. The final CHICAGO asthma discharge tool, called the CAPE (CHICAGO Action Plan after ED discharge) (Figure 4) , has a Flesch–Kincaid reading level of 4.7 (compared with existing documents in use across the CHICAGO ED clinical centers that had reading levels of 5.5–7.8). The final discharge tool is also designed to support simplified reading strategies by using multiple principles of information design, such as typographic hierarchy to create priority and aid browsing, use of an underlying grid and white space to reduce visual complexity, reduced line length and text blocks to increase readability and the

24

J. Comp. Eff. Res. (2016) 5(1)

option for printing or copying in black and white for low-cost distribution. Phase III: evaluate stakeholder preferences

Results of the Gibson survey suggested caregiver and clinician preference for the new discharge tool across all categories (Figure 5) . Qualitative assessments using the card sorting activity also demonstrated a preference for the new discharge tool (Figure 6) . Among clinicians, the card sorting activity elicited strong emotional responses as they shared their frustrations over their current asthma action plans. There was full consensus for the new tool for four of eleven questions (items 3, 4, 6 and 10). Clinicians reported that they favored the new tool in part because of its clarity; one physician observed that “for our patient population, this is awesome…it is not too dense, and it is not word overloaded. This is very succinct. It is easy to understand.” Clinicians also indicated that the new tool could serve as a support for conversations and for teaching selfmanagement. One clinician noted that the new tool was ‘more interactive, so as you go through it people will have a visual and can ask questions’, whereas when using the existing document ‘there is not as much to go through, step-wise, so its less likely that you are going to be able to point and continue a dialog based on what is in the document’. Caregiver responses also demonstrated a preference for the new discharge tool (Figure 7). However, caregiver responses were strongest for the new tool when queried about behavioral items, such as “This document is more clear about which actions I should take after the Emergency Department” (item 3) or “I am more likely to

future science group

Engaging stakeholders to design a comparative effectiveness trial in children with uncontrolled asthma 

Overall format

Asthma discharge plan

• • • •

Four simple steps to organize action, conversation Written in second person to increase personal relevance Strong typographic hierarchy and underlying grid to organize content; Simplified language and reduced medical jargon (Flesch–Kincaid reading grade level of 4.7) • Illustrations average 50% of content and visual space

1

Your oral steroid is:

Things to know: How much How often For how long It is very important you complete the dosage

Your “rescue” medicine is:

Inhaler

Spacer

Mask

Nebulizer

For how long

• Should be used only if your child is having symptoms during an asthma attack/ with symptoms

Take every day EVEN IF no visibile symptoms Mask

Mark your meds at the pharmacy:

green sticker for “controller” medicine

• may be allergy medication, such as Singulair and Associate

Nebulizer

2

See your child’s doctor within 3 days of your ER visit

Doctor’s name

3

4

Read the signs

no coughing or wheezing

Illustrations by Paula Falco

GREEN ZONE

Even if your child shows no signs of breathing problems, keep using the “controller” medicine every day.

Stay on top of asthma

Call your child’s regular doctor as soon as possible to help you understand your child’s asthma and home treatment plan.

Identify your child’s asthma triggers

Build a trigger list of what seems to make your child’s asthma act up. Add to that list as you notice new triggers. Try to help your child avoid these! If your child has a cold, use your child’s action plan; and help them to blow their nose. Avoid smoking—a known asthma trigger—and avoid having your child in a house where someone smokes.

plays as usual sleeps soundly

YELLOW ZONE

Call doctor

coughs a lot at night

If your child shows any of these signs, use “rescue” medicine right away, keep using “controller” medicine, and call your doctor.

hurts to breathe deeply

breathes fast when standing in place

breathing does not get better within 20 minutes of taking “rescue” medicine

Your appointment date and time

Clinic telephone number

Don’t wait! Call with questions

peak flow is at normal level

breathes easily

Here are some examples of common asthma triggers:

What are your child’s triggers?

hard to sleep because of breathing problems

Give medications as prescribed

Review how to use the inhalers with your child’s doctor. Develop tricks to help remind you to give the medications. What might be useful tricks? set an alert on your smartphone keep medicine by your coffee pot

Call 911 breathing so hard that they are drowsy or sleepy lips or fingernails are grey or blue

RED ZONE

Get help If your child has any of these signs, use “rescue” medicine, and go to the emergency room or call 911.

breathing gets worse within 20 minutes of taking “rescue” medicine ribs show when breathing

hard time saying a full sentence without a breath

Take your child to the doctor regularly

Your child’s doctor is there to help—they want to see how well your child is doing and to review your child’s symptom control. Together you and your doctor will discuss a new Asthma Home Plan, with instructions for when your child’s asthma is under control and when it is not well-controlled.

hard time breathing when sitting in place

hard time walking

• QR code (bottom) links smart phones to more online asthma resources • Step-by-step instruction to help caregivers understand proper inhaler technique • Simple language and illustration helps kids teach themselves and siblings

• examples include Pulmicort, Flovent, Azmacort, Advair

Other:

Go play

A ‘smart’ document

How often

“rescue” medicine

• is typically albuterol with a name like: Proventil, Pro-Air, Ventolin, Xopenex

Things to know: 1st dose time/date

Spacer

Inhaler

• if you were given these in the emergency room, it is very important that you finish them!

Mark your meds Things to know: • should be used only if your at the pharmacy: child is having symptoms during an asthma attack/ red sticker for with symptoms

1 dose time/date Number of puffs How often st

Medication clarification

Date

• is another powerful “rescue” medicine

1st dose time/date Liquid

Pills

Number of puffs

• Stoplight structure to organize action, signal the dangerous progression of asthma • Illustrations to diagnose child’s status; • Friendly style to engage kids • Call-outs to reinforce illustrations – brief and written in jargon-free language

Today’s date

Take your asthma medicine

Your “controller” medicine is:

Symptom recognition + action steps

Child’s name

Doctor’s signature

After that, use ONLY when symptoms occur

• Medication instruction broken down into clear steps • Illustrations of tools to clarify/teach unfamiliar terms • Checkboxes required so as to trigger discussion with caregiver • Colored stickers provided to distinguish inhalers

Research Article

How to use an inhaler with a spacer Works as well as nebulizer!

Trigger recognition + self-management tips

Take cap off the inhaler. Check for and remove any dust, lint, or other objects. Shake the inhaler well.

Attach the inhaler to the spacer.

Breathe out all the air, away from the spacer.

Put lips around device, press inhaler one time. This puts one puff of medicine into the spacer.

Breathe in deeply and slowly, and hold your breath.

Remove the device from the mouth. Then hold your breath for 5 secs. Then breathe normally away from the spacer.

If your child needs to take another puff of medicine, wait 1 minute. After one minute, repeat steps 3 to 6.

Rinsing is only necessary if the medicine you just took was an inhaled steroid. Have your child rinse his or her mouth out with water after the last puff of medicine. Make sure your child spits the water out. Do not allow the child to swallow the water. Recap the inhaler.

• Illustrations of triggers for clinicians to point at to aid conversation • Room fo caregivers – not just doctors – to write so as to promote caregiver role in asthma control • Home environment questions to prompt caregiver engagement • Tips and tricks to help start new self-management practices

From the American College of Chest Physicians Illustrations by Paula Falco

© 2015 CHICAGO Plan investigators + PCOR1 The CHICAGO Plan is a PCOR1-funded study comparing asthma interventions. For questions regarding this document or the CHICAGO Plan. contact: Trevonne Thompson, MD. [email protected]

What is this? This is a QR code. To use it, go to the app store on your smartphone, search for ‘QR code readers’ and download the free app. To learn more about asthma, scan this code with the app to go directly to the Respiratory Health Association website. Or go to the link below: www.tinyurl.com/asthmalib

Figure 4. The new discharge tool, or CAPE (CHICAGO Action Plan after emergancy discharge), includes several key advances in design to improve communication of medications, symptom recognition and action steps, and trigger recognition and selfmanagement. CHICAGO: Coordinated Healthcare Interventions for Childhood Asthma Gaps in Outcome; QR: Quick response.

future science group

www.futuremedicine.com

25

Research Article  Erwin, Martin, Flippin et al.

Gibson survey results Average scores per category Highest possible score = 5 1

2

3

4

5

1

2

3

4

4.8

Overall

4.8 3.9

3.1 4.4

Appearance

4.7

2.3

3.5 4.6

Usefulness

5

4.6 3.8

3.2 4.7

Content

3.4 Clinician document scores (nurses and physicians; n = 12)

Caregiver document scores (n = 8)

CHICAGO discharge tool Existing discharge tool

Figure 5. Using the Gibson survey,clinicians and caregivers report higher levels of preference for the new discharge tool, CAPE (CHICAGO Action Plan after ED discharge), compared with existing documents. CHICAGO: Coordinated Healthcare Interventions for Childhood Asthma Gaps in Outcome.

hang this up in my home” (item 4) or “I would prefer to receive this document in the emergency room” (item 5). Overall document preference was more equivocal when statements raised issues that could be interpreted as relating to their competency, such as “This document is harder to read” or “This document is more confusing.” The research team also encountered differences in language use, with one participant, for example, selecting the new tool as ‘more overwhelming’, as in overwhelmingly good. Such responses raise important issues about the wording of questions to fit target populations. Discussion In this report, we presented the methods and outcomes of a three-phase process that employed various methods of contextual inquiry in the design of CAPE, a novel stakeholder-balanced asthma action plan for high-risk African–American and Latino children with uncontrolled asthma for use on ED discharge. Several aspects of CAPE are noteworthy, including the use of a design that maximizes visual learning and individualization. More important than the final tool, however, is the process employed to create it. The formative process presented here demonstrates a new multistakeholder driven methodology to inform the development of interventions suitable for comparative effectiveness research. This method

26

J. Comp. Eff. Res. (2016) 5(1)

is novel because it shifts the design process from one that relies exclusively on teams of medical experts who focus on content to multidisciplinary teams with expertise in uncovering the context of use to inform content requirements, which may also help to overcome barriers to implementation in real-world clinical and nonclinical settings. Contextual inquiry differs from standard qualitative inquiry because it is designed to immerse the investigator in the everyday practices of the informant, rather than asking the informant to recall their experiences, as standard interview or focus group methods do. With informants and investigators both working in situ, informants reveal with their behavior what they often fail to recall when asked, and investigators build firsthand experience with how informants engage in the activity being studied. For the CAPE, as an example, contextual inquiry revealed that complex conversations between stakeholders were taking place unsupported by existing materials. Designing for conversation guided content development and format. We are now testing the effectiveness of the asthma discharge tool on implementation and clinical outcomes in the multicenter PCORI-funded CHICAGO comparative effectiveness trial. Comparative effectiveness research is intended to be a response to the expressed needs of end-users about which interventions are most effective for patients

future science group

Engaging stakeholders to design a comparative effectiveness trial in children with uncontrolled asthma 

under specific circumstances [1] . To our knowledge, this is the first report to engage end-users in contextual inquiry in designing asthma action tools for children, their caregivers and clinicians providing medical care

Research Article

in EDs. Contextual inquiry is an important addition to comparative effectiveness research because it creates a new form of evidence. By locating the research team in the context of use, data is collected not just about

Card sorting scores: clinicians Aggregated clinician responses (n = 12) 1

2

3

4

5

6 Nurse responses (max = 6)

1. I feel more confident using this document

Physician responses (max = 6)

2. I think this document is more likely to facilitate constructive conversations between Emergency Department clinicians and caregivers 3. I would prefer to use this document with my patients 4. I would rather use the visuals in this document to communicate essential information to my patients in the emergency department 5. I think this document is more respectful of my time with my patient 6. I think this document provides more guidance for my patients after discharge 7. I think this document is more accessible for caregivers with low reading levels 8. I think this document is more effective in helping my patients understand their medications 9. I think this document represents a more standardized protocol for care 10. I think this document will have greater impact on my patient’s health

11. I feel this is a more serious medical document

Document A (CHICAGO discharge tool) Document B (existing tool) Both documents Neither document

Figure 6. The card sorting activitysupports findings from the Gibson survey, indicating greater clinician preference for the new discharge tool, CAPE (CHICAGO Action Plan after emergency department discharge). CHICAGO: Coordinated Healthcare Interventions for Childhood Asthma Gaps in Outcome.

future science group

www.futuremedicine.com

27

Research Article  Erwin, Martin, Flippin et al.

Card sorting scores: caregivers Aggregated caregiver responses (n = 8) 1

2

3

4

5

6

7

8 Responses

1. I think that with this document, my family would visit the emergency department less

Positive assessment statements; higher scores are better

2. This document is easier to read 3. This document is more clear about which action I should take first after the emergency department 4. I am more likely to hang this document up in my home 5. If I was in the emergency room with my child, I would prefer to receive this document 6. This document is more approachable

7. This document is more organized

8. This document is more worthy of my time

9. This document is better for my family

Negative assessment statements; lower scores are better

10. This document is more overwhelming

11. This document is more confusing

Document A (CHICAGO discharge tool) Document B (existing tool) Both documents Neither document Figure 7. The card sorting activitysupports findings from the Gibson survey,indicating greater caregiver preference for the new discharge tool. CHICAGO: Coordinated Healthcare Interventions for Childhood Asthma Gaps in Outcome

patient practices, but how those practices are shaped by interactions with others, how practices become embedded (or not) in everyday activities and responsibilities and how practices are negotiated against the inevitable disruptors of everyday life that compete for patients’ time and attention. This evidence is especially productive when seeking to tailor interventions to populations and circumstances for better uptake and adherence. Additionally, contextual inquiry is equally productive when applied to ED clinical staff, who report struggling

28

J. Comp. Eff. Res. (2016) 5(1)

with tools and protocols that do not seem designed to fit the complex operational realities of the ED and its many users. Indeed, results of quantitative and qualitative assessments suggest substantial preference among clinical staff for the new tool’s ease of use and fit with desired caregiver conversations, compared with existing tools used in the ED. By design, our new discharge tool shifts the communication paradigm in the ED from delivery of information (from expert to novice) to supporting collaborative

future science group

Engaging stakeholders to design a comparative effectiveness trial in children with uncontrolled asthma 

conversation (between equally engaged stakeholders). This is an important shift because it institutionalizes what some clinicians in this study have already acknowledged: that communication is more effective when built with caregivers through an exchange of information in the moment than ‘delivered’ to them in a monologue or handout. The new discharge tool was noted by clinical staff as a tool that ‘at least starts the conversation’, ‘opens up the conversation better’ and ‘is more interactive as you go through it with people’. Shifting to a collaborative model in the ED also creates new conditions that encourage caregivers to see themselves as partners in asthma control. Current ED experiences contribute to caregiver belief that an asthma attack is an event best fixed by a doctor in an ED; a collaborative approach can stress asthma as a chronic condition best managed by caregivers at home. While our project offers multiple strengths, our approach to engagement was limited to addressing the needs of elementary school-age high-risk African–American and Latino children with uncontrolled asthma presenting to the ED. This focus was deliberately given the goals of our PCORI-funded CHICAGO comparative effectiveness trial. However, the discharge tool may not be sufficiently optimized for other populations (e.g., preschoolers, adolescents, adults). While we did include physicians and nurses, our engagement process did not include other clinicians who provide care to children with uncontrolled asthma (e.g., respiratory therapists, social workers, pharmacists). Moreover, most of our children and their caregivers were low income and African– American. The methods employed in this project offer a template for future work that could address the needs of other end-user populations, including those with other conditions (e.g., diabetes, hypertension, sickle cell disease). Conclusion We present the application of contextual inquiry of enduser stakeholders to design a novel ED-based asthma action plan (CAPE) for a comparative effectiveness trial

Research Article

in asthma. We speculate that engaging multiple stakeholders in the design of a discharge tool ‘fit for purpose’ offers a promising approach for improving asthma selfmanagement skills and improving asthma outcomes in African–American and Latino children presenting to the ED for uncontrolled asthma. Our on-going PCORIfunded CHICAGO comparative effectiveness trial will evaluate the effects of employing CAPE (vs usual care) in the ED on implementation and clinical outcomes. Acknowledgements The authors gratefully acknowledge the help of T MacTavish, IIT Institute of Design; JT Senko, JH Stroger, Jr Hospital of Cook County; CJ Lohff, Chicago Department of Public Health; ZE Pittsenbarger, Ann and Robert H Lurie Children’s Hospital of Chicago; J E Kramer, Rush University Medical Center; H Margellos-Anast, LS Zun, Mount Sinai Hospital; SM Paik and J Solway, University of Chicago; ML Berbaum, N Bracken, and HA Gussin, for their role in the development of the CHICAGO comparative effectiveness trial. The authors thank L Sanker for her help in the development and conduct of focus groups. The authors also thank the families and the staff in the CHICAGO ED clinical centers and partner organizations who contributed their time to make this study possible.

Financial & competing interests disclosure The study was sponsored by the Patient-Centered Outcomes Research Institute (contract #AS-1307-05420). The authors have no other relevant affiliations or financial involvement with any organization or entity with a financial interest in or financial conflict with the subject matter or materials discussed in the manuscript apart from those disclosed. No writing assistance was utilized in the production of this manuscript.

Ethical conduct of research The authors state that they have obtained appropriate institutional review board approval or have followed the principles outlined in the Declaration of Helsinki for all human or animal experimental investigations. In addition, for investigations involving human subjects, informed consent has been obtained from the participants involved.

Executive summary Designing interventions to improve communication in comparative effectiveness research • Multistakeholder driven design methods of contextual inquiry can be successfully employed to inform the development of interventions for comparative effectiveness research. • Our approach shifted the design process from one that relies exclusively on teams of medical experts who focus on content to multidisciplinary teams with expertise in uncovering the context of use to inform content requirements, which may also help to overcome barriers to implementation in real-world clinical and nonclinical settings. • The new discharge tool or CAPE (CHICAGO Action Plan after emergency department discharge) shifts the communicationparadigm in the emergency department from delivery of information (from expert to novice) to supporting collaborative conversation (between equally-engaged stakeholders).

future science group

www.futuremedicine.com

29

Research Article  Erwin, Martin, Flippin et al. emergency department visits for children with asthma. Pediatrics 107(4), 706–711 (2001).

References 1

Comparative Effectiveness Research: National Institutes of Health. PCAST Session on Health Reform and CER (2009). www.whitehouse.gov

2

National Institutes of Health/National Heart, Lung, and Blood Institute. Expert Panel Report 3: guidelines on asthma. www.nhlbi.nih.gov/health-pro

3

Global Initiative for Asthma (GINA). www.ginasthma.org/

4

Gupta S, Wan FT, Ducharme FM, Chignell MH, Lougheed MD, Straus SE. Asthma action plans are highly variable and do not conform to best visual design practices. Ann. Allergy Asthma Immunol. 108(4), 260–265 (2012).

5

Ring N, Jepson R, Hoskins G, et al. Understanding what helps or hinders asthma action plan use: a systematic review and synthesis of the qualitative literature. Patient Educ. Couns. 85(2), e131–e143 (2011).

Beyer H, Holtzblatt K. Contextual design. Interactions 6 (1), 32–42 (1999).

16

Suchman L. Plans and Situated Actions: The Problem of Human-Machine Communication. Cambridge University Press, Cambridge, UK (1987).

17

Glaser B, Strauss A. The discovery of grounded theory: strategies for qualitative research. Aldine de Gruyter, NY, USA (1967).

18

Corbin J, Strauss A. Grounded theory research: procedures, canons, and evaluative criteria. Qual. Sociol. 13(1), 3–21 (1990).

19

Seligman HK, Wallace AS, DeWalt DA et al. Facilitating behavior change with low-literacy patient education materials. Am. J. Health Behav. 31(0 1), S69–S78 (2007).

20

Doak CC, Doak LG, Root JH. Teaching Patients with Low Literacy Skills (Second Edition). J.B. Lippincott Company, PA, USA (1996).

21

Doak CC, Doak LG, Friedell GH, Meade CD. Improving comprehension for cancer patients with low literacy skills: strategies for clinicians. CA. Cancer J. Clin. 48(3), 151–162 (1998).

6

Patient-Centered Outcomes Research Institute. www.pcori.org/research-results/2013 

7

Gupta RS, Zhang X, Sharp LK, Shannon JJ, Weiss KB. Geographic variability in childhood asthma prevalence in Chicago. J. Allergy Clin. Immunol. 121(3), 639–645 (2008).

8

Martin B, Hanington B. Universal Methods of Design: 100 Ways to Research Complex Problems, Develop Innovative Ideas and Design Effective Solutions. Rockport Publishers, MA, USA (2012).

22

Oates D J, Paasche Orlow MK. Health literacy: communication strategies to improve patient comprehension of cardiovascular health. Circulation 119(7), 1049–1051 (2009).

9

Proctor E, Silmere H, Raghavan R et al. Outcomes for implementation research: conceptual distinctions, measurement challenges, and research agenda. Adm. Policy Ment. Health. 38(2), 65–76 (2011).

23

Osborn CY, Cavanaugh K, Kripalani S. Strategies to address low health literacy and numeracy in diabetes. Clin. Diabetes 28(4), 171–175 (2010).

24

10

Eggleston PA, Diette G, Lipsett M et al. Lessons learned for the study of childhood asthma from the Centers for Children’s Environmental Health and Disease Prevention Research. Environ Health Perspect. 113(10), 1430–1436 (2005).

U.S. Department of Health and Human Services. Plain language: a promising strategy for clearly communicating health information and improving health literacy. www.health.gov/communication/literacy

25

11

Swartz LJ, Callahan KA, Butz AM et al. Methods and issues in conducting a community-based environmental randomized trial. Environ. Res. 95(2), 156–165 (2004).

Gibson PA, Ruby C, Craig MD. A health/patient education database for family practice. Bull. Med. Libr. Assoc. 79(4), 357–369 (1991).

26

12

Eggleston PA, Butz A, Rand C et al. Home environmental intervention in inner-city asthma: a randomized controlled clinical trial. Ann. Allergy Asthma Immunol. 95(6), 518–524 (2005).

Cataldo EF, Johnson RM, Kellstedt LA, Milbrath LW. Card sorting as a technique for survey interviewing. Pub. Opin. Quart. 34(2), 202–215 (1970).

27

Nawaz A. A comparison of card sorting analysis methods. Presented at: The 10th Asia Pacific Conference on Computer Human Interaction (Apchi 2012). Matsue-city, Shimane, Japan, 28–31 August 2012.

28

Erwin K, Pollari T. Small packages for big (qualitative) data. In: EPIC: The Ethnographic Practice In Industry Conference. John Wiley & Sons, Hoboken, NJ, USA, 44–61 (2013).

13

14

30

15

Al-Muhsen S, Horanieh N, Dulgom S et al. Poor asthma education and medication compliance are associated with increased emergency department visits by asthmatic children. Ann Thorac Med. 10(2), 123–131 (2015). Adams RJ, Fuhlbrigge A, Finkelstein JA et al. Impact of inhaled anti-inflammatory therapy on hospitalization and

J. Comp. Eff. Res. (2016) 5(1)

future science group