GENERAL MEDICINE/ORIGINAL RESEARCH
Measuring Patient Tolerance for Future Adverse Events in Low-Risk Emergency Department Chest Pain Patients Jennifer C. Chen, MD, MPH; Richelle J. Cooper, MD, MSHS; Ana Lopez-O’Sullivan, MD; David L. Schriger, MD, MPH
Study objective: We assess emergency department (ED) patients’ risk thresholds for preferring admission versus discharge when presenting with chest pain and determine how the method of information presentation affects patients’ choices. Methods: In this cross-sectional survey, we enrolled a convenience sample of lower-risk acute chest pain patients from an urban ED. We presented patients with a hypothetical value for the risk of adverse outcome that could be decreased by hospitalization and asked them to identify the risk threshold at which they preferred admission versus discharge. We randomized patients to a method of numeric presentation (natural frequency or percentage) and the initial risk presented (low or high) and followed each numeric assessment with an assessment based on visually depicted risks. Results: We enrolled 246 patients and analyzed data on 234 with complete information. The geometric mean risk threshold with numeric presentation was 1 in 736 (1 in 233 with a percentage presentation; 1 in 2,425 with a natural frequency presentation) and 1 in 490 with a visual presentation. Fifty-nine percent of patients (137/234) chose the lowest or highest risk values offered. One hundred fourteen patients chose different thresholds for numeric and visual risk presentations. We observed strong anchoring effects; patients starting with the lowest risk chose a lower threshold than those starting with the highest risk possible and vice versa. Conclusion: Using an expected utility model to measure patients’ risk thresholds does not seem to work, either to find a stable risk preference within individuals or in groups. Further work in measurement of patients’ risk tolerance or methods of shared decisionmaking not dependent on assessment of risk tolerance is needed. [Ann Emerg Med. 2014;-:1-10.] Please see page XX for the Editor’s Capsule Summary of this article. 0196-0644/$-see front matter Copyright © 2014 by the American College of Emergency Physicians. http://dx.doi.org/10.1016/j.annemergmed.2013.12.025
SEE EDITORIAL, P.
.
---
INTRODUCTION Background Despite advances in medical knowledge and diagnostic technologies, it remains challenging to accurately identify the few patients who have a non-ST elevation myocardial infarction or acute coronary syndrome from the many other patients who present with chest pain. Physicians’ practices aimed at minimizing the risk of adverse outcomes in low-probability, high-morbidity conditions such as chest pain likely maximize the interests of physicians and hospitals but may not optimize the interests of patients or society.1 Most studies on decisionmaking in the emergency department (ED) focus on physician factors such as fear of malpractice, personality (eg, risk aversion), and tolerance of uncertainty.2-4 ED patients are rarely queried about their desires in regard to care options, their tolerance for future bad outcomes, or their willingness to undergo treatment to decrease the probability of a future bad outcome when that risk is already low. Volume
-,
no.
-
:
-
2014
Importance Understanding individual patients’ risk thresholds is an important part of achieving shared decisionmaking in which, in partnership with a physician, patients consider diagnostic and management options, communicate their preferences, and help select the course of action that best fits their preferences.5 Many questions remain about how to implement the shared decisionmaking model in EDs or other settings with high amounts of uncertainty and time pressure. Research on best practices in risk communication has generally focused on preventive health activities such as screening or less time-sensitive outpatient decisions such as medication management, rather than on acute medical care.6 Several theoretical models exist to describe how patients make decisions about their health care. Classic decision theory involves an economic expected utility model, in which a rational actor chooses from a set of well-defined options based on an assessment of the probability and severity of outcomes.7 Cognitive psychology literature shows that decisionmaking frequently deviates from expected utility modeling in predictable ways because of heuristics and biases such as framing effects.8 Annals of Emergency Medicine 1
Chen et al
Measuring Patient Tolerance for Future Adverse Events
Editor’s Capsule Summary
What is already known on this topic Rational choice models of decisionmaking that seek to maximize the patient’s utility require elicitation of the patient’s willingness to accept risk. This assessment may be highly sensitive to the manner in which it is acquired. What question this study addressed This study randomized 234 low-risk chest pain patients to elicitations of risk as percentages or natural frequencies, starting with the lowest risk or highest risk, followed by elicitation of risk using visual depictions of risk. What this study adds to our knowledge Individuals’ risk thresholds varied between numeric and visual elicitations of the same information. Thresholds also varied greatly across patients; almost half chose the lowest risk threshold, particularly when it was presented first. How this is relevant to clinical practice If current methods cannot elicit stable risk preferences, then rational choice models of decisionmaking will not function. Other models of shared decisionmaking not dependent on assessment of risk preferences should be explored.
However, one review on risk communication in health decisions showed that framing effects observed in laboratory cognitive psychology studies are not found as reliably in clinical studies of risk communication9 A pilot study by Brown et al10 assessed risk tolerance in patients presenting to the ED with chest pain. The authors presented to each patient an initial hypothetical level of risk of an acute adverse event and asked whether he or she would prefer admission to the hospital or discharge from the ED. They then increased or decreased the risk level until the patient reached a point of indifference. Patients chose a wide range of risks at which they were indifferent to admission or discharge (median value 6.5%; interquartile range 0.5% to 22.9%). This heterogeneity of risk preferences seemed to support neither a rational choice theory nor a heuristics and biases model and differed substantively from other trials of ED patients with chest pain that found more homogeneous risk preferences.11-13 Brown et al10 used the term “risk tolerance” to refer to a measurement of a preference for a “risk” versus “risk plus inconvenience” level, as opposed to a more conventional definition as an individual’s willingness to accept a risk or hazard. We continued to use the term “risk tolerance” as in the study by Brown et al.10 2 Annals of Emergency Medicine
Goals of This Investigation The failure of the article by Brown et al10 to identify homogeneous beliefs about risk preferences may be explained in 2 ways. The theoretical framework, expected utility theory, could be correct while their specific implementation was problematic, or expected utility theory is not a good model. In an attempt to differentiate these 2 explanations, we adapted the pilot study by Brown et al10 to control for some of its potential shortcomings by using both numeric and visual risk presentations, standardizing the hypothetical risks, including an assessment of patient comprehension, and adding consideration of levels of risk below 1%. We explored when chest pain patients reached a point of indifference to hospitalization to reduce the risk of a bad outcome, which we will refer to as the “patient’s risk threshold” throughout this article. To focus in on the role of risk communication, we decided to present data in a variety of formats, both numeric and graphic. Few existing studies compare numeric and graphic versus numeric information only in health care decisions14 despite evidence that shows patients prefer more data rather than less15 and have poor understanding of numeric probabilities,16 and that graphic displays may increase risk communication effectiveness.17,18 In addition, multiple complementary formats may be the most appropriate.19 We also wanted to evaluate the anchoring effect of starting values on patients’ stated risk thresholds and self-reported comprehension of the exercise.
MATERIALS AND METHODS Study Design and Setting In this cross-sectional survey, we enrolled patients presenting with acute chest pain to a single urban ED during convenient times between February 2009 and October 2010. We randomized participants to one of 4 survey versions, which varied by whether numeric risks were presented as percentages or natural frequencies (eg, 1 in 1,000) and whether the survey began with high or low risk values (Figure 1). We chose to present numeric risks in 2 ways to explore the effect of the heuristic of using natural frequencies compared with percentage format.20,21 The hospital research committee and institutional review board approved this study. The West Los Angeles Veterans Affairs (VA) Medical Center is a 300-bed urban teaching hospital that serves area veterans. The annual ED census is approximately 26,000; 95% of patients are men. We chose this setting because it is an integrated health care system with low or no out-of-pocket costs for medications, ED visits, and hospitalizations, thereby reducing variation in risk threshold because of the effects of health insurance status and wealth. Selection of Participants Patients who presented with a chief complaint of acute chest pain were eligible for this study if they were aged 18 years or older, were designated for inpatient admission, and had a negative initial ultrasensitive troponin test result; did not have an Volume
-,
no.
-
:
-
2014
Chen et al
Measuring Patient Tolerance for Future Adverse Events
Low to high group follow this arrow
Risk of a Bad Outcome* Percentage (%) Frequency If If If If Discharged Admied Discharged Admied 0.01 0.009 1 in 10,000 1 in 11,000 0.1 0.09 1 in 1,000 1 in 1,100 0.2 0.018 1 in 500 1 in 550 1 0.9 1 in 100 1 in 110 2 1.8 1 in 50 1 in 55 5 4.5 1 in 20 1 in 22 10 9 1 in 10 1 in 11 20 18 1 in 5 1 in 5.5 33 29.7 1 in 3 1 in 3.3 50 45 1 in 2 1 in 2.2
High to low group follow this arrow
Figure 2. Probability ladder. In this hypothetical scenario, we set the benefit with hospitalization as a 10% reduction in risk of adverse events in the 2 weeks after the ED visit. *Bad outcome was defined for the patient as a heart attack, emergency heart surgery/angioplasty, heart failure, or death, occurring in the 2 weeks after the index ED visit (see Figure E1, available online at http://www.annemergmed.com).
Figure 1. Randomization of patients. RA, Research assistant.
ECG result consistent with ST-elevation myocardial infarction or other acute ischemia, active severe chest pain (>7/10), systolic blood pressure less than 90 mm Hg or greater than 180 mm Hg, diastolic blood pressure greater than 110 mm Hg, Thrombolysis in Myocardial Infarction (TIMI) risk score greater than 3 of 7, pregnancy, cognitive impairment, or altered mental status; and were not prisoners, VA employees, non-English speaking, on a psychiatric hold, or deemed medically unstable in the opinion of the treating physician. We enrolled all patients during 4-hour blocks when research assistants were available. Research assistants underwent standardized group training and periodic refresher meetings about the research protocol. Each research assistant was individually proctored at bedside by an investigator for several patient encounters before working independently. We used a computer-generated randomization sequence to assign enrolled patients to one of 4 combinations of numeric format and probability direction, ie, percentage, low to high; natural frequency, low to high; percentage, high to low; and natural frequency, high to low (Figure 1). We Volume
-,
no.
-
:
-
2014
performed the randomization a priori and created sequentially numbered opaque survey packets. During intermittent, 4-hour shifts between 8 AM and midnight, typically on weekdays, the research assistant identified every potentially eligible patient and obtained informed consent. With the help of the treating physician, she then calculated a TIMI score and confirmed eligibility. The patient interview began with the research assistant’s reading the risk tolerance script that explained the exercise (Figure E1, available online at http://www.annemergmed.com) and presenting the patient with hypothetical probabilities of an adverse outcome with and without hospitalization. The patient was given a copy of the risk script to refer to and reread as needed during the exercise. This risk script included a description in lay language of possible adverse clinical outcomes such as heart failure, and instructions to the patient to also consider financial costs and other opportunity costs such as lost work time. The probability of an adverse outcome with hospitalization was always set 10% lower than the probability without hospitalization. The randomization determined the initial value of the probability and the form in which it was presented. The patient was told the first set of probabilities (as either natural frequencies or percentages) and asked whether he or she would (theoretically) prefer hospitalization or discharge. For example, a patient randomized to the “start with low probabilities described as natural frequencies” group would be asked, “If you knew the risk of something bad happening to you related to your heart was 1 in 10,000 and admitting you to the hospital would decrease your risk to 1 in 11,000, would you want to be admitted to the hospital?” If the patient said yes, this phase was complete. If the patient said no, the research assistant would repeat the scenario, increasing the risk per the probability ladder (Figure 2). Scenarios were repeated with progressive levels of risk until the patient changed his or her preference for admission versus discharge, the numeric risk threshold. Annals of Emergency Medicine 3
Chen et al
Measuring Patient Tolerance for Future Adverse Events The research assistant then followed a script to confirm the numeric risk threshold (Appendix E1, available online at http://www.annemergmed.com) and rated how much effort (on a 5-point scale) was required to get the patient to reach a stable transition point and whether she thought the patient understood the exercise. Each patient also self-rated comprehension. Patients traversed the probability ladder in the same direction a second time, using visual representations of risk as conveyed by a 10,000-dot array (Appendix E2, available online at http:// www.annemergmed.com). Research assistants showed the patients 2 dot arrays, one showing the risk with hospitalization, the other without, for each level of risk on the probability ladder in a side-by-side comparison until the point of indifference was reached. After determining and confirming the visual risk threshold, research assistants and patients used the same ratings questions to assess the effort and comprehension of the visual risk portion of the exercise. Finally, the research assistant collected demographic information and a structured cardiac history from the patients. The entire exercise took about 20 minutes to complete. Research assistants were instructed to adhere to the script and use “probability” or “likelihood” when talking about risk, not “percentage” or “frequency.” All patients were admitted (the decision was made by the treating physician before study enrollment), and as part of the consent process it was explained to the patient that their treatment was unaffected by the survey. Data Collection and Processing Research assistants recorded results on standardized data forms and entered the deidentified data into a secure Web-based instrument that wrote into a Microsoft Access (Microsoft, Redmond, WA) database. Outcome Measures The primary outcome was the patients’ risk threshold, as determined by the numeric and visual exercises. Secondary outcomes included patient and research assistants’ ratings of the patients’ comprehension. Primary Data Analysis We decided a priori on a target sample size of at least 50 patients per randomization arm, a number we believed sufficient to provide adequate precision for our estimation of risk thresholds. We conducted a purely descriptive data analysis with Stata (version 12; StataCorp, College Station, TX), excluding the few cases with missing data for the numeric or visual primary outcome measures. We examined the distributions of the numeric risk threshold and the visual risk threshold on log scales, assessing central tendency with the geometric mean, and graphed each patient’s numeric and visual threshold in paired fashion. We then stratified data according to randomization variables (numeric format, low or high risks presented first), comprehension variables, and demographic 4 Annals of Emergency Medicine
variables to look for anchoring effects and other explanatory factors.
RESULTS Characteristics of Study Subjects We estimate that 2,550 patients were admitted for initial troponin-negative rule-out acute coronary syndrome from March 2009 through August 2010, and 360 of them were available for enrollment when a research assistant was present (Figure E2, available online at http://www.annemergmed.com). Thirty-six patients were ineligible after initial screening. We approached 311 patients for enrollment. Fifty-one patients declined to participate, and 14 patients for various other reasons were not enrolled. We randomized 246 patients and analyzed the 234 for whom we had complete data on their numeric and visual risk thresholds. The Table presents the enrolled patients’ demographic and clinical information. Ninety-six percent of study patients were men, 41% of subjects were white, another 41% were black. The median age was 60 years (interquartile range 54 to 69 years). Sixty-six percent of the patients had an annual household income of $25,000 or less. The geometric mean for the risk threshold based on numeric presentation was 1 in 736 (Figure 3A contains 95% CIs for all risks). The geometric mean for the risk threshold based on visual presentation was 1 in 490. Fifty-nine percent of patients (137/ 234) chose the lowest or highest risk we offered them, which hereafter we will refer to as “extreme values.” In addition to analyzing the risk threshold across all patients, we separately analyzed those who chose extreme values and those who did not (Figure 3B). Almost half (n¼115) the patients desired hospitalization even when the risk of a bad outcome was 1 in 10,000 or less without hospitalization, and regardless of whether risks were presented as numbers (49% wanted admission) or graphics (42%) (Figure 4). Ten percent of patients wanted to be discharged even when the risk of a bad outcome was 1 in 2 or higher (numeric 9%; visual 11%). For the 97 patients who selected a nonextreme numeric risk threshold, the geometric mean was 1 in 56 (Figure 3B). When data were presented as a visual display, 110 patients selected a nonextreme value for their risk threshold, with geometric mean of 1 in 61. Seventy-six patients chose a nonextreme value on both the visual and numeric exercises; their geometric means were 1 in 54 for the numeric depiction and 1 in 39 for the visual depiction (Figure 3C). Results were strongly influenced by whether patients were offered the lowest or highest risk first, with patients offered the lowest risk first anchoring on lower-risk thresholds (Figures 3A and 5). Hospitalization at the lowest possible risk level was desired by 65% of patients (77/119) whose interview began with the lowest risk level (1 in 10,000, or .01%) and by 33% of patients (38/115) whose interview began with the highest risk level (1 in 2, or 50%). Similarly, discharge was desired at the highest risk level (1 in 2, or 50%) by 17% (19/115) of those offered the highest risk first and 3% of patients (3/119) offered Volume
-,
no.
-
:
-
2014
Chen et al
Measuring Patient Tolerance for Future Adverse Events
Table. Baseline characteristics of patients.*
Variable Age, median (IQR) Sex (% male) Race, No. (%) White Black Other/unknown Annual household income, $ (%) 25,000 25,001–50,000 50,001–75,000 75,001–100,000 100,001 Decline to state Marital status Single Married Other (divorced, widowed, separated) Highest educational level Some high school Completed high school Some college College degree or more Previous myocardial infarction History of cardiac diagnostic procedure Treadmill Angiogram PCI CABG Previous overnight ED or inpatient evaluation of chest pain Yes No Self-assessment of “having heart trouble today” Yes No Maybe Self-assessment of health status Below average Average Above average Excellent TIMI score 0–1 2–7
Low to High, Percentage (n[60)
High to Low, Percentage (n[57)
Low to High, Probability (n[59)
High to Low, Probability (n[58)
All Patients (n[234)
59 (54–69) 95
59 (54–65) 98
62 (50–73) 93
61 (54–66) 97
60 (54–69) 96
25 (42) 28 (47) 7 (12)
26 (46) 20 (35) 11 (19)
28 (47) 19 (32) 12 (20)
18 (31) 30 (52) 10 (17)
97 (41) 97 (41) 40 (17)
40 5 6 1 1 7
(67) (8) (10) (2) (2) (12)
40 (70) 7 (12) 5 (9) 0 0 5 (9)
40 7 3 1 2 6
35 9 1 3 2 8
17 (28) 19 (32) 24 (40)
19 (33) 15 (26) 22 (39)
21 (36) 17 (29) 21 (36)
12 (21) 19 (33) 27 (47)
4 15 26 15 18
(7) (25) (43) (25) (30)
8 16 22 11 14
(14) (28) (39) (19) (25)
3 6 33 15 13
(5) (10) (56) (25) (22)
6 12 26 14 10
(10) (21) (28) (24) (17)
21 49 107 55 55
(9) (21) (45) (24) (24)
9 9 8 7
(15) (15) (13) (12)
6 9 7 7
(11) (16) (16) (12)
11 14 6 4
(19) (24) (10) (7)
12 9 8 5
(21) (16) (14) (8)
38 41 28 23
(16) (18) (12) (10)
(68) (12) (5) (2) (3) (10)
(60) (16) (2) (5) (3) (14)
155 28 15 5 5 26
(66) (12) (6) (2) (2) (11)
69 (29) 70 (30) 94 (40)
34 (57) 26 (43)
28 (49) 29 (51)
33 (56) 26 (44)
30 (52) 28 (48)
125 (53) 109 (47)
32 (53) 14 (23) 13 (22)
24 (42) 17 (30) 15 (26)
27 (46) 18 (30) 13 (22)
23 (40) 26 (45) 9 (15)
106 (45) 75 (32) 50 (21)
22 25 12 1
20 19 13 3
25 19 9 5
(42) (32) (15) (8)
19 (33) 20 (34) 18 (31) 0
27 (46) 32 (54)
32 (55) 26 (45)
(37) (42) (20) (1)
29 (48) 31 (52)
(35) (33) (23) (5)
28 (49) 29 (51)
86 83 52 9
(37) (35) (22) (4)
116 (50) 118 (50)
IQR, Interquartile range. *Data are presented as No. (%) unless otherwise indicated.
the lowest risk first. For the 97 patients who did not choose an extreme value, there remained an anchoring effect, with those randomized to start with the lowest risk value ending with a threshold lower than the group who started with the highest risk value (Figure 3B). The format of the numeric presentation (percentage versus natural frequency) strongly influenced mean risk threshold, with patients offered percentages choosing higher thresholds (1 in 233 versus 1 in 2,425) (Figures 3A and 5). When presented risk probabilities depicted as percentages, 39% of patients (46/117) desired hospitalization at the lowest risk level and 14% (16/117) Volume
-,
no.
-
:
-
2014
desired discharge at the highest risk level. When presented risks reported as natural frequencies, 59% of patients (69/117) desired hospitalization at the lowest risk level and 5% (6/117) desired discharge at the highest risk level. The 2 effects (numeric format and anchoring) were more than additive and less than multiplicative. Sixty-nine percent of patients (41/59) given the lowest risk first as a frequency (eg, 1 in 10,000) wanted hospitalization at that level, whereas only 18% of patients (10/57) given the highest risk first as a percentage (50%) wanted hospitalization at the 1 in 10,000 level (Figure 5). Annals of Emergency Medicine 5
Chen et al
Measuring Patient Tolerance for Future Adverse Events
A Entire sample
Method of presentation of risk Numeric Visual Direction Numeric Low to High High to Low Visual Low to High High to Low Numeric format Frequency Percent
B
Lower CI 1 in X 1193 792
Upper CI 1 in X 454 303
119 115
3561 144
6018 293
2107 71
119 115
2190 104
3885 204
1235 536
117 117
2425 233
4367 455
1347 110 1 in 10,000
1 in 1,000
1 in 100
1 in 10
Subgroups who chose non-extreme values
Method of presentation of risk Numeric Visual
Geometric mean risk threshold N 1 in X 97 56 110 61
Direction Numeric Low to High High to Low Visual Low to High High to Low Numeric format Frequency Percent
C
Geometric mean risk threshold N 1 in X 234 736 234 490
Lower CI 1 in X 93 96
Upper CI 1 in X 34 38
39 58
210 23
302 39
88 14
44 66
219 26
466 42
103 16
42 55
209 21
509 32
86 13
Subgroup who chose non-extreme values for both numeric and visual presentations
Numeric Visual
76 76
54 39
93 62
31 25
1 in 10,000
1 in 1,000
1 in 100
1 in 10
Figure 3. A, Geometric mean for the risk threshold stratified by sample and risk presentation. B, Subgroups who chose nonextreme values.
One hundred fourteen patients had different risk thresholds for the numeric and visual risk presentations (Figure 6). One hundred ten patients as opposed to 97 chose a nonextreme value when risks were depicted visually. Twenty-nine patients changed from the lowest risk value to a higher risk threshold when shown a visual depiction of the risk. This means 25% 6 Annals of Emergency Medicine
(29/115) of those who wanted to be hospitalized at the lowest risk (1 in 10,000, or .01%) with numeric depiction chose a higher risk threshold value when shown pictures. Eight of the 22 patients (36%) who selected the highest risk value with the numeric depiction changed to a lower risk threshold when shown the visual depiction. Volume
-,
no.
-
:
-
2014
Chen et al
Measuring Patient Tolerance for Future Adverse Events
Figure 4. Histograms for patient risk threshold by numeric and visual risk. The histograms depict the risk threshold, the risk of a bad outcome without hospitalization at or above which the patient stated he or she would want to be hospitalized. The rightmost bar in each histogram represents the patients who wanted to be discharged even when the risk of a bad outcome was 1 in 2. The leftmost bars represent patients who desired hospitalization at the lowest risk offered, 1 in 10,000. The blue lines represent the geometric mean risk threshold for all patients; the red lines, for patients who chose a risk threshold other than the 2 extreme values of 1 in 2 or 1 in 10,000 (n¼97 for the numeric and n¼110 for the visual). Risk thresholds in the “numeric” panel were obtained by verbal interview, using numeric representations of probability (percentages or frequencies). Thresholds in the “visual” panel were obtained by showing a page with 10,000 dots, with dots colored red representing bad outcomes (Appendix E2, available online at http://www.annemergmed.com).
Forty of the 97 patients who chose a nonextreme value during the numeric exercise chose a higher threshold, 37 chose a lower threshold, and 20 did not change their value after seeing visual depictions of the risk. Distributions of risk threshold stratified on demographic variables, such as race, highest level of education, and insurance status, and clinical variables such as general health and history of cardiac disease did not reveal any strong patterns to justify making conclusions (Figures E3 to E14, available online at http://www.annemergmed.com). We analyzed comprehension by categorizing anyone who rated that they “agree” or “strongly agree” with the statement of confidence as confident. The patients reported they were confident about their comprehension of both the numeric (89%) and visual exercises (93%). Research assistants also rated patients’ comprehension of the numeric (70%) and visual exercises (79%) favorably. There were no important differences in confidence ratings based on the method of numeric probability presentation or whether risks were presented low to high or high to low. Patient self-ratings were unrelated to whether the patient chose an extreme threshold or an intermediate one or to the value of the Volume
-,
no.
-
:
-
2014
Figure 5. Histogram of risk threshold by direction and numeric probability format. The histograms depict the risk threshold—the risk of a bad outcome without hospitalization at or above which the patients stated they would want to be hospitalized—stratified by whether the exercise began with high or low risk values and whether numeric risks were presented as percentages or frequencies. The leftmost bar in each histogram represents patients who desired hospitalization at the lowest risk offered, 1 in 10,000; the rightmost, those who desired discharge even when the risk of a bad outcome was 1 in 2 or higher. The blue lines depict the geometric mean for each histogram for all patients; the red lines, the geometric mean for those patients who chose a value other than the 2 extremes.
threshold chosen (Figure E15A, available online at http://www. annemergmed.com). Research assistants tended to have higher confidence in patients who chose intermediate values for thresholds and to those who chose higher risk thresholds (Figure E15B, available online at http://www.annemergmed. com). Because almost all the patients self-rated as confident and almost all the research assistants rated as confident their confidence levels in the patients’ comprehension, there was insufficient heterogeneity in patient-research assistant agreement on comprehension at the individual patient level to suggest any patterns.
LIMITATIONS A major limitation of this study is that we cannot know with certainty whether patients chose risk thresholds that accurately reflect their true beliefs. Patients’ self-ratings of comprehension were high but they may have been reluctant to admit they were confused or did not know what they did not know. Furthermore, we used the underlying assumptions: (1) a single stable risk preference exists and is measurable, and (2) the net expected utility/threshold model is an appropriate model for decisionmaking. Threats to internal validity include the use of a hypothetical scenario, the failure to adjust the benefits of hospitalization to Annals of Emergency Medicine 7
Chen et al
Measuring Patient Tolerance for Future Adverse Events
Each Patient's Numeric and Visual Thresholds Grouped by Numeric Threshold (N=234) 1 in 2 1 in 10
1 in 100
1 in 1,000
1 in 10,000 Lowest (n=115)
Patients
Intermediate (n=97)
Highest (n=22)
Figure 6. Change in patient risk threshold between numeric and visual formats. Each patient is represented by a vertical line that extends from his numeric risk threshold to his visual risk threshold. If the 2 are the same, the patient is represented by a dot. Patients are organized by their numeric values, starting with the 115 patients (in blue) who desired hospitalization at the lowest risk offered (1 in 10,000). The next group (red) is composed of the 97 patients who chose an intermediate numeric value, further sorted by whether the visual threshold was higher (n¼40), unchanged (n¼20), or lower (n¼37) than the numeric threshold. The final group (black) is 22 patients who wanted discharge even at the highest numeric risk offered (1 in 2). Readers should consider whether the change from numeric to visual was small (representing fine tuning of one’s opinion) or spans several orders of magnitude (suggesting that the 2 methods produced significantly different opinions).
each patient’s clinical circumstance, and the assumption that hospitalization would reduce morbidity by 10%. We included explicit mention of financial and opportunity costs to hospitalization in the risk tolerance script but did not include situational factors such as living alone or barriers to transportation to and from the hospital for outpatient evaluation or recurrent symptoms. We decided to represent probabilities as percentages, natural frequencies, and a visual array and to present them separately so we could study differences among them. Different results might be obtained if other methods were used to represent risk and if representations of risk had been presented simultaneously. Finally, our decision to separately examine patients who chose extreme values and those who chose intermediate ones is predicated on the assumption that those who chose extreme values did so because they were unwilling to engage in the exercise as opposed to because these extreme values represented their belief. We may also have introduced bias into the study by always having the visual risk come after the numeric risk presentation because fatigue, unwillingness to abandon one’s initial commitment to a number, or other factors may have 8 Annals of Emergency Medicine
contaminated choices made with the visual scale. It may also be that risk preferences are inherently not stable. Some naturalistic decisionmaking models emphasize social context, social roles, and role-appropriate behaviors as drivers of decisionmaking.22,23 Other sociological models of risk describe how worldviews and cultural and social institutions play a strong role in generating normative frameworks of risk, such as with conceptions of risk about nuclear energy, strongly implying that risk measurement is unstable and heterogeneous.24 Finally, psychological factors such as trust in the treating physician and in health care institutions, and health beliefs about disease course and cure that affect feelings of self-efficacy may affect risk preferences.25,26 Additionally, research assistants may have consciously or unconsciously influenced the respondents. The students did not wear white coats or other hospital uniforms and introduced themselves as student research assistants as opposed to clinicians. Although we attempted to reduce the effect of deference to authority figures by having student research assistants rather than physician researchers perform the semistructured interviews, we acknowledge that novices (ie, the patients) often defer to authority figures (ie, the medical system) for decisionmaking. Selection bias represents the greatest threat to external validity. We conducted this study at a VA hospital, which has a patient population that is different from that of the general ED community in demographics and case mix. In particular, the total number of female participants was very low. Given differences between men and women about risk perception and risk taking, making generalizations beyond male patients’ risk preferences is problematic.27,28 We chose the VA setting to minimize the role of financial and other access considerations. It is possible that some veterans have a bias toward resource use because they may view health care as a benefit already earned. This bias would lead to strong preferences for hospitalization even at very low levels of risk. We conducted the study at convenient times but minimized selection bias by including nearly all eligible patients during those times. We have no reason to believe that patients treated on weekends, night shifts, and summer months, the times underrepresented in our study, should be different from those we captured.
DISCUSSION The main conclusion from our study is that using an expected utility model to measure patients’ risk thresholds does not seem to work, either to find a stable risk preference within individuals or in groups. Although theoretically there may be an elusive combination of the right words and pictures to minimize noise and bias introduced by assessment tools, it is doubtful, given the heterogeneity found within our study and several other studies discussed below. The majority of our patients desired hospitalization when it reduced risk from 1 in 10,000 to 1 in 11,000, an implied number needed to treat to benefit of 110,000. Other studies with different designs and measurement techniques reported higher risk thresholds. A similar study that Volume
-,
no.
-
:
-
2014
Chen et al
Measuring Patient Tolerance for Future Adverse Events
used the same risk tolerance script but initially offered patients intermediate risk values expressed as percentages found that patients had a median risk threshold of 1 in 15.10 Davis et al11 demonstrated that the majority of ED patients with chest pain who were offered a hypothetical scenario in which hospitalization decreased the risk of complications of myocardial infarction from 1 in 100 to 1 in 500 preferred outpatient management (number needed to treat to benefit
Measuring Patient Tolerance for Future Adverse Events in Low-Risk Emergency Department Chest Pain Patients Jennifer C. Chen, MD, MPH; Richelle J. Cooper, MD, MSHS; Ana Lopez-O’Sullivan, MD; David L. Schriger, MD, MPH
Study objective: We assess emergency department (ED) patients’ risk thresholds for preferring admission versus discharge when presenting with chest pain and determine how the method of information presentation affects patients’ choices. Methods: In this cross-sectional survey, we enrolled a convenience sample of lower-risk acute chest pain patients from an urban ED. We presented patients with a hypothetical value for the risk of adverse outcome that could be decreased by hospitalization and asked them to identify the risk threshold at which they preferred admission versus discharge. We randomized patients to a method of numeric presentation (natural frequency or percentage) and the initial risk presented (low or high) and followed each numeric assessment with an assessment based on visually depicted risks. Results: We enrolled 246 patients and analyzed data on 234 with complete information. The geometric mean risk threshold with numeric presentation was 1 in 736 (1 in 233 with a percentage presentation; 1 in 2,425 with a natural frequency presentation) and 1 in 490 with a visual presentation. Fifty-nine percent of patients (137/234) chose the lowest or highest risk values offered. One hundred fourteen patients chose different thresholds for numeric and visual risk presentations. We observed strong anchoring effects; patients starting with the lowest risk chose a lower threshold than those starting with the highest risk possible and vice versa. Conclusion: Using an expected utility model to measure patients’ risk thresholds does not seem to work, either to find a stable risk preference within individuals or in groups. Further work in measurement of patients’ risk tolerance or methods of shared decisionmaking not dependent on assessment of risk tolerance is needed. [Ann Emerg Med. 2014;-:1-10.] Please see page XX for the Editor’s Capsule Summary of this article. 0196-0644/$-see front matter Copyright © 2014 by the American College of Emergency Physicians. http://dx.doi.org/10.1016/j.annemergmed.2013.12.025
SEE EDITORIAL, P.
.
---
INTRODUCTION Background Despite advances in medical knowledge and diagnostic technologies, it remains challenging to accurately identify the few patients who have a non-ST elevation myocardial infarction or acute coronary syndrome from the many other patients who present with chest pain. Physicians’ practices aimed at minimizing the risk of adverse outcomes in low-probability, high-morbidity conditions such as chest pain likely maximize the interests of physicians and hospitals but may not optimize the interests of patients or society.1 Most studies on decisionmaking in the emergency department (ED) focus on physician factors such as fear of malpractice, personality (eg, risk aversion), and tolerance of uncertainty.2-4 ED patients are rarely queried about their desires in regard to care options, their tolerance for future bad outcomes, or their willingness to undergo treatment to decrease the probability of a future bad outcome when that risk is already low. Volume
-,
no.
-
:
-
2014
Importance Understanding individual patients’ risk thresholds is an important part of achieving shared decisionmaking in which, in partnership with a physician, patients consider diagnostic and management options, communicate their preferences, and help select the course of action that best fits their preferences.5 Many questions remain about how to implement the shared decisionmaking model in EDs or other settings with high amounts of uncertainty and time pressure. Research on best practices in risk communication has generally focused on preventive health activities such as screening or less time-sensitive outpatient decisions such as medication management, rather than on acute medical care.6 Several theoretical models exist to describe how patients make decisions about their health care. Classic decision theory involves an economic expected utility model, in which a rational actor chooses from a set of well-defined options based on an assessment of the probability and severity of outcomes.7 Cognitive psychology literature shows that decisionmaking frequently deviates from expected utility modeling in predictable ways because of heuristics and biases such as framing effects.8 Annals of Emergency Medicine 1
Chen et al
Measuring Patient Tolerance for Future Adverse Events
Editor’s Capsule Summary
What is already known on this topic Rational choice models of decisionmaking that seek to maximize the patient’s utility require elicitation of the patient’s willingness to accept risk. This assessment may be highly sensitive to the manner in which it is acquired. What question this study addressed This study randomized 234 low-risk chest pain patients to elicitations of risk as percentages or natural frequencies, starting with the lowest risk or highest risk, followed by elicitation of risk using visual depictions of risk. What this study adds to our knowledge Individuals’ risk thresholds varied between numeric and visual elicitations of the same information. Thresholds also varied greatly across patients; almost half chose the lowest risk threshold, particularly when it was presented first. How this is relevant to clinical practice If current methods cannot elicit stable risk preferences, then rational choice models of decisionmaking will not function. Other models of shared decisionmaking not dependent on assessment of risk preferences should be explored.
However, one review on risk communication in health decisions showed that framing effects observed in laboratory cognitive psychology studies are not found as reliably in clinical studies of risk communication9 A pilot study by Brown et al10 assessed risk tolerance in patients presenting to the ED with chest pain. The authors presented to each patient an initial hypothetical level of risk of an acute adverse event and asked whether he or she would prefer admission to the hospital or discharge from the ED. They then increased or decreased the risk level until the patient reached a point of indifference. Patients chose a wide range of risks at which they were indifferent to admission or discharge (median value 6.5%; interquartile range 0.5% to 22.9%). This heterogeneity of risk preferences seemed to support neither a rational choice theory nor a heuristics and biases model and differed substantively from other trials of ED patients with chest pain that found more homogeneous risk preferences.11-13 Brown et al10 used the term “risk tolerance” to refer to a measurement of a preference for a “risk” versus “risk plus inconvenience” level, as opposed to a more conventional definition as an individual’s willingness to accept a risk or hazard. We continued to use the term “risk tolerance” as in the study by Brown et al.10 2 Annals of Emergency Medicine
Goals of This Investigation The failure of the article by Brown et al10 to identify homogeneous beliefs about risk preferences may be explained in 2 ways. The theoretical framework, expected utility theory, could be correct while their specific implementation was problematic, or expected utility theory is not a good model. In an attempt to differentiate these 2 explanations, we adapted the pilot study by Brown et al10 to control for some of its potential shortcomings by using both numeric and visual risk presentations, standardizing the hypothetical risks, including an assessment of patient comprehension, and adding consideration of levels of risk below 1%. We explored when chest pain patients reached a point of indifference to hospitalization to reduce the risk of a bad outcome, which we will refer to as the “patient’s risk threshold” throughout this article. To focus in on the role of risk communication, we decided to present data in a variety of formats, both numeric and graphic. Few existing studies compare numeric and graphic versus numeric information only in health care decisions14 despite evidence that shows patients prefer more data rather than less15 and have poor understanding of numeric probabilities,16 and that graphic displays may increase risk communication effectiveness.17,18 In addition, multiple complementary formats may be the most appropriate.19 We also wanted to evaluate the anchoring effect of starting values on patients’ stated risk thresholds and self-reported comprehension of the exercise.
MATERIALS AND METHODS Study Design and Setting In this cross-sectional survey, we enrolled patients presenting with acute chest pain to a single urban ED during convenient times between February 2009 and October 2010. We randomized participants to one of 4 survey versions, which varied by whether numeric risks were presented as percentages or natural frequencies (eg, 1 in 1,000) and whether the survey began with high or low risk values (Figure 1). We chose to present numeric risks in 2 ways to explore the effect of the heuristic of using natural frequencies compared with percentage format.20,21 The hospital research committee and institutional review board approved this study. The West Los Angeles Veterans Affairs (VA) Medical Center is a 300-bed urban teaching hospital that serves area veterans. The annual ED census is approximately 26,000; 95% of patients are men. We chose this setting because it is an integrated health care system with low or no out-of-pocket costs for medications, ED visits, and hospitalizations, thereby reducing variation in risk threshold because of the effects of health insurance status and wealth. Selection of Participants Patients who presented with a chief complaint of acute chest pain were eligible for this study if they were aged 18 years or older, were designated for inpatient admission, and had a negative initial ultrasensitive troponin test result; did not have an Volume
-,
no.
-
:
-
2014
Chen et al
Measuring Patient Tolerance for Future Adverse Events
Low to high group follow this arrow
Risk of a Bad Outcome* Percentage (%) Frequency If If If If Discharged Admied Discharged Admied 0.01 0.009 1 in 10,000 1 in 11,000 0.1 0.09 1 in 1,000 1 in 1,100 0.2 0.018 1 in 500 1 in 550 1 0.9 1 in 100 1 in 110 2 1.8 1 in 50 1 in 55 5 4.5 1 in 20 1 in 22 10 9 1 in 10 1 in 11 20 18 1 in 5 1 in 5.5 33 29.7 1 in 3 1 in 3.3 50 45 1 in 2 1 in 2.2
High to low group follow this arrow
Figure 2. Probability ladder. In this hypothetical scenario, we set the benefit with hospitalization as a 10% reduction in risk of adverse events in the 2 weeks after the ED visit. *Bad outcome was defined for the patient as a heart attack, emergency heart surgery/angioplasty, heart failure, or death, occurring in the 2 weeks after the index ED visit (see Figure E1, available online at http://www.annemergmed.com).
Figure 1. Randomization of patients. RA, Research assistant.
ECG result consistent with ST-elevation myocardial infarction or other acute ischemia, active severe chest pain (>7/10), systolic blood pressure less than 90 mm Hg or greater than 180 mm Hg, diastolic blood pressure greater than 110 mm Hg, Thrombolysis in Myocardial Infarction (TIMI) risk score greater than 3 of 7, pregnancy, cognitive impairment, or altered mental status; and were not prisoners, VA employees, non-English speaking, on a psychiatric hold, or deemed medically unstable in the opinion of the treating physician. We enrolled all patients during 4-hour blocks when research assistants were available. Research assistants underwent standardized group training and periodic refresher meetings about the research protocol. Each research assistant was individually proctored at bedside by an investigator for several patient encounters before working independently. We used a computer-generated randomization sequence to assign enrolled patients to one of 4 combinations of numeric format and probability direction, ie, percentage, low to high; natural frequency, low to high; percentage, high to low; and natural frequency, high to low (Figure 1). We Volume
-,
no.
-
:
-
2014
performed the randomization a priori and created sequentially numbered opaque survey packets. During intermittent, 4-hour shifts between 8 AM and midnight, typically on weekdays, the research assistant identified every potentially eligible patient and obtained informed consent. With the help of the treating physician, she then calculated a TIMI score and confirmed eligibility. The patient interview began with the research assistant’s reading the risk tolerance script that explained the exercise (Figure E1, available online at http://www.annemergmed.com) and presenting the patient with hypothetical probabilities of an adverse outcome with and without hospitalization. The patient was given a copy of the risk script to refer to and reread as needed during the exercise. This risk script included a description in lay language of possible adverse clinical outcomes such as heart failure, and instructions to the patient to also consider financial costs and other opportunity costs such as lost work time. The probability of an adverse outcome with hospitalization was always set 10% lower than the probability without hospitalization. The randomization determined the initial value of the probability and the form in which it was presented. The patient was told the first set of probabilities (as either natural frequencies or percentages) and asked whether he or she would (theoretically) prefer hospitalization or discharge. For example, a patient randomized to the “start with low probabilities described as natural frequencies” group would be asked, “If you knew the risk of something bad happening to you related to your heart was 1 in 10,000 and admitting you to the hospital would decrease your risk to 1 in 11,000, would you want to be admitted to the hospital?” If the patient said yes, this phase was complete. If the patient said no, the research assistant would repeat the scenario, increasing the risk per the probability ladder (Figure 2). Scenarios were repeated with progressive levels of risk until the patient changed his or her preference for admission versus discharge, the numeric risk threshold. Annals of Emergency Medicine 3
Chen et al
Measuring Patient Tolerance for Future Adverse Events The research assistant then followed a script to confirm the numeric risk threshold (Appendix E1, available online at http://www.annemergmed.com) and rated how much effort (on a 5-point scale) was required to get the patient to reach a stable transition point and whether she thought the patient understood the exercise. Each patient also self-rated comprehension. Patients traversed the probability ladder in the same direction a second time, using visual representations of risk as conveyed by a 10,000-dot array (Appendix E2, available online at http:// www.annemergmed.com). Research assistants showed the patients 2 dot arrays, one showing the risk with hospitalization, the other without, for each level of risk on the probability ladder in a side-by-side comparison until the point of indifference was reached. After determining and confirming the visual risk threshold, research assistants and patients used the same ratings questions to assess the effort and comprehension of the visual risk portion of the exercise. Finally, the research assistant collected demographic information and a structured cardiac history from the patients. The entire exercise took about 20 minutes to complete. Research assistants were instructed to adhere to the script and use “probability” or “likelihood” when talking about risk, not “percentage” or “frequency.” All patients were admitted (the decision was made by the treating physician before study enrollment), and as part of the consent process it was explained to the patient that their treatment was unaffected by the survey. Data Collection and Processing Research assistants recorded results on standardized data forms and entered the deidentified data into a secure Web-based instrument that wrote into a Microsoft Access (Microsoft, Redmond, WA) database. Outcome Measures The primary outcome was the patients’ risk threshold, as determined by the numeric and visual exercises. Secondary outcomes included patient and research assistants’ ratings of the patients’ comprehension. Primary Data Analysis We decided a priori on a target sample size of at least 50 patients per randomization arm, a number we believed sufficient to provide adequate precision for our estimation of risk thresholds. We conducted a purely descriptive data analysis with Stata (version 12; StataCorp, College Station, TX), excluding the few cases with missing data for the numeric or visual primary outcome measures. We examined the distributions of the numeric risk threshold and the visual risk threshold on log scales, assessing central tendency with the geometric mean, and graphed each patient’s numeric and visual threshold in paired fashion. We then stratified data according to randomization variables (numeric format, low or high risks presented first), comprehension variables, and demographic 4 Annals of Emergency Medicine
variables to look for anchoring effects and other explanatory factors.
RESULTS Characteristics of Study Subjects We estimate that 2,550 patients were admitted for initial troponin-negative rule-out acute coronary syndrome from March 2009 through August 2010, and 360 of them were available for enrollment when a research assistant was present (Figure E2, available online at http://www.annemergmed.com). Thirty-six patients were ineligible after initial screening. We approached 311 patients for enrollment. Fifty-one patients declined to participate, and 14 patients for various other reasons were not enrolled. We randomized 246 patients and analyzed the 234 for whom we had complete data on their numeric and visual risk thresholds. The Table presents the enrolled patients’ demographic and clinical information. Ninety-six percent of study patients were men, 41% of subjects were white, another 41% were black. The median age was 60 years (interquartile range 54 to 69 years). Sixty-six percent of the patients had an annual household income of $25,000 or less. The geometric mean for the risk threshold based on numeric presentation was 1 in 736 (Figure 3A contains 95% CIs for all risks). The geometric mean for the risk threshold based on visual presentation was 1 in 490. Fifty-nine percent of patients (137/ 234) chose the lowest or highest risk we offered them, which hereafter we will refer to as “extreme values.” In addition to analyzing the risk threshold across all patients, we separately analyzed those who chose extreme values and those who did not (Figure 3B). Almost half (n¼115) the patients desired hospitalization even when the risk of a bad outcome was 1 in 10,000 or less without hospitalization, and regardless of whether risks were presented as numbers (49% wanted admission) or graphics (42%) (Figure 4). Ten percent of patients wanted to be discharged even when the risk of a bad outcome was 1 in 2 or higher (numeric 9%; visual 11%). For the 97 patients who selected a nonextreme numeric risk threshold, the geometric mean was 1 in 56 (Figure 3B). When data were presented as a visual display, 110 patients selected a nonextreme value for their risk threshold, with geometric mean of 1 in 61. Seventy-six patients chose a nonextreme value on both the visual and numeric exercises; their geometric means were 1 in 54 for the numeric depiction and 1 in 39 for the visual depiction (Figure 3C). Results were strongly influenced by whether patients were offered the lowest or highest risk first, with patients offered the lowest risk first anchoring on lower-risk thresholds (Figures 3A and 5). Hospitalization at the lowest possible risk level was desired by 65% of patients (77/119) whose interview began with the lowest risk level (1 in 10,000, or .01%) and by 33% of patients (38/115) whose interview began with the highest risk level (1 in 2, or 50%). Similarly, discharge was desired at the highest risk level (1 in 2, or 50%) by 17% (19/115) of those offered the highest risk first and 3% of patients (3/119) offered Volume
-,
no.
-
:
-
2014
Chen et al
Measuring Patient Tolerance for Future Adverse Events
Table. Baseline characteristics of patients.*
Variable Age, median (IQR) Sex (% male) Race, No. (%) White Black Other/unknown Annual household income, $ (%) 25,000 25,001–50,000 50,001–75,000 75,001–100,000 100,001 Decline to state Marital status Single Married Other (divorced, widowed, separated) Highest educational level Some high school Completed high school Some college College degree or more Previous myocardial infarction History of cardiac diagnostic procedure Treadmill Angiogram PCI CABG Previous overnight ED or inpatient evaluation of chest pain Yes No Self-assessment of “having heart trouble today” Yes No Maybe Self-assessment of health status Below average Average Above average Excellent TIMI score 0–1 2–7
Low to High, Percentage (n[60)
High to Low, Percentage (n[57)
Low to High, Probability (n[59)
High to Low, Probability (n[58)
All Patients (n[234)
59 (54–69) 95
59 (54–65) 98
62 (50–73) 93
61 (54–66) 97
60 (54–69) 96
25 (42) 28 (47) 7 (12)
26 (46) 20 (35) 11 (19)
28 (47) 19 (32) 12 (20)
18 (31) 30 (52) 10 (17)
97 (41) 97 (41) 40 (17)
40 5 6 1 1 7
(67) (8) (10) (2) (2) (12)
40 (70) 7 (12) 5 (9) 0 0 5 (9)
40 7 3 1 2 6
35 9 1 3 2 8
17 (28) 19 (32) 24 (40)
19 (33) 15 (26) 22 (39)
21 (36) 17 (29) 21 (36)
12 (21) 19 (33) 27 (47)
4 15 26 15 18
(7) (25) (43) (25) (30)
8 16 22 11 14
(14) (28) (39) (19) (25)
3 6 33 15 13
(5) (10) (56) (25) (22)
6 12 26 14 10
(10) (21) (28) (24) (17)
21 49 107 55 55
(9) (21) (45) (24) (24)
9 9 8 7
(15) (15) (13) (12)
6 9 7 7
(11) (16) (16) (12)
11 14 6 4
(19) (24) (10) (7)
12 9 8 5
(21) (16) (14) (8)
38 41 28 23
(16) (18) (12) (10)
(68) (12) (5) (2) (3) (10)
(60) (16) (2) (5) (3) (14)
155 28 15 5 5 26
(66) (12) (6) (2) (2) (11)
69 (29) 70 (30) 94 (40)
34 (57) 26 (43)
28 (49) 29 (51)
33 (56) 26 (44)
30 (52) 28 (48)
125 (53) 109 (47)
32 (53) 14 (23) 13 (22)
24 (42) 17 (30) 15 (26)
27 (46) 18 (30) 13 (22)
23 (40) 26 (45) 9 (15)
106 (45) 75 (32) 50 (21)
22 25 12 1
20 19 13 3
25 19 9 5
(42) (32) (15) (8)
19 (33) 20 (34) 18 (31) 0
27 (46) 32 (54)
32 (55) 26 (45)
(37) (42) (20) (1)
29 (48) 31 (52)
(35) (33) (23) (5)
28 (49) 29 (51)
86 83 52 9
(37) (35) (22) (4)
116 (50) 118 (50)
IQR, Interquartile range. *Data are presented as No. (%) unless otherwise indicated.
the lowest risk first. For the 97 patients who did not choose an extreme value, there remained an anchoring effect, with those randomized to start with the lowest risk value ending with a threshold lower than the group who started with the highest risk value (Figure 3B). The format of the numeric presentation (percentage versus natural frequency) strongly influenced mean risk threshold, with patients offered percentages choosing higher thresholds (1 in 233 versus 1 in 2,425) (Figures 3A and 5). When presented risk probabilities depicted as percentages, 39% of patients (46/117) desired hospitalization at the lowest risk level and 14% (16/117) Volume
-,
no.
-
:
-
2014
desired discharge at the highest risk level. When presented risks reported as natural frequencies, 59% of patients (69/117) desired hospitalization at the lowest risk level and 5% (6/117) desired discharge at the highest risk level. The 2 effects (numeric format and anchoring) were more than additive and less than multiplicative. Sixty-nine percent of patients (41/59) given the lowest risk first as a frequency (eg, 1 in 10,000) wanted hospitalization at that level, whereas only 18% of patients (10/57) given the highest risk first as a percentage (50%) wanted hospitalization at the 1 in 10,000 level (Figure 5). Annals of Emergency Medicine 5
Chen et al
Measuring Patient Tolerance for Future Adverse Events
A Entire sample
Method of presentation of risk Numeric Visual Direction Numeric Low to High High to Low Visual Low to High High to Low Numeric format Frequency Percent
B
Lower CI 1 in X 1193 792
Upper CI 1 in X 454 303
119 115
3561 144
6018 293
2107 71
119 115
2190 104
3885 204
1235 536
117 117
2425 233
4367 455
1347 110 1 in 10,000
1 in 1,000
1 in 100
1 in 10
Subgroups who chose non-extreme values
Method of presentation of risk Numeric Visual
Geometric mean risk threshold N 1 in X 97 56 110 61
Direction Numeric Low to High High to Low Visual Low to High High to Low Numeric format Frequency Percent
C
Geometric mean risk threshold N 1 in X 234 736 234 490
Lower CI 1 in X 93 96
Upper CI 1 in X 34 38
39 58
210 23
302 39
88 14
44 66
219 26
466 42
103 16
42 55
209 21
509 32
86 13
Subgroup who chose non-extreme values for both numeric and visual presentations
Numeric Visual
76 76
54 39
93 62
31 25
1 in 10,000
1 in 1,000
1 in 100
1 in 10
Figure 3. A, Geometric mean for the risk threshold stratified by sample and risk presentation. B, Subgroups who chose nonextreme values.
One hundred fourteen patients had different risk thresholds for the numeric and visual risk presentations (Figure 6). One hundred ten patients as opposed to 97 chose a nonextreme value when risks were depicted visually. Twenty-nine patients changed from the lowest risk value to a higher risk threshold when shown a visual depiction of the risk. This means 25% 6 Annals of Emergency Medicine
(29/115) of those who wanted to be hospitalized at the lowest risk (1 in 10,000, or .01%) with numeric depiction chose a higher risk threshold value when shown pictures. Eight of the 22 patients (36%) who selected the highest risk value with the numeric depiction changed to a lower risk threshold when shown the visual depiction. Volume
-,
no.
-
:
-
2014
Chen et al
Measuring Patient Tolerance for Future Adverse Events
Figure 4. Histograms for patient risk threshold by numeric and visual risk. The histograms depict the risk threshold, the risk of a bad outcome without hospitalization at or above which the patient stated he or she would want to be hospitalized. The rightmost bar in each histogram represents the patients who wanted to be discharged even when the risk of a bad outcome was 1 in 2. The leftmost bars represent patients who desired hospitalization at the lowest risk offered, 1 in 10,000. The blue lines represent the geometric mean risk threshold for all patients; the red lines, for patients who chose a risk threshold other than the 2 extreme values of 1 in 2 or 1 in 10,000 (n¼97 for the numeric and n¼110 for the visual). Risk thresholds in the “numeric” panel were obtained by verbal interview, using numeric representations of probability (percentages or frequencies). Thresholds in the “visual” panel were obtained by showing a page with 10,000 dots, with dots colored red representing bad outcomes (Appendix E2, available online at http://www.annemergmed.com).
Forty of the 97 patients who chose a nonextreme value during the numeric exercise chose a higher threshold, 37 chose a lower threshold, and 20 did not change their value after seeing visual depictions of the risk. Distributions of risk threshold stratified on demographic variables, such as race, highest level of education, and insurance status, and clinical variables such as general health and history of cardiac disease did not reveal any strong patterns to justify making conclusions (Figures E3 to E14, available online at http://www.annemergmed.com). We analyzed comprehension by categorizing anyone who rated that they “agree” or “strongly agree” with the statement of confidence as confident. The patients reported they were confident about their comprehension of both the numeric (89%) and visual exercises (93%). Research assistants also rated patients’ comprehension of the numeric (70%) and visual exercises (79%) favorably. There were no important differences in confidence ratings based on the method of numeric probability presentation or whether risks were presented low to high or high to low. Patient self-ratings were unrelated to whether the patient chose an extreme threshold or an intermediate one or to the value of the Volume
-,
no.
-
:
-
2014
Figure 5. Histogram of risk threshold by direction and numeric probability format. The histograms depict the risk threshold—the risk of a bad outcome without hospitalization at or above which the patients stated they would want to be hospitalized—stratified by whether the exercise began with high or low risk values and whether numeric risks were presented as percentages or frequencies. The leftmost bar in each histogram represents patients who desired hospitalization at the lowest risk offered, 1 in 10,000; the rightmost, those who desired discharge even when the risk of a bad outcome was 1 in 2 or higher. The blue lines depict the geometric mean for each histogram for all patients; the red lines, the geometric mean for those patients who chose a value other than the 2 extremes.
threshold chosen (Figure E15A, available online at http://www. annemergmed.com). Research assistants tended to have higher confidence in patients who chose intermediate values for thresholds and to those who chose higher risk thresholds (Figure E15B, available online at http://www.annemergmed. com). Because almost all the patients self-rated as confident and almost all the research assistants rated as confident their confidence levels in the patients’ comprehension, there was insufficient heterogeneity in patient-research assistant agreement on comprehension at the individual patient level to suggest any patterns.
LIMITATIONS A major limitation of this study is that we cannot know with certainty whether patients chose risk thresholds that accurately reflect their true beliefs. Patients’ self-ratings of comprehension were high but they may have been reluctant to admit they were confused or did not know what they did not know. Furthermore, we used the underlying assumptions: (1) a single stable risk preference exists and is measurable, and (2) the net expected utility/threshold model is an appropriate model for decisionmaking. Threats to internal validity include the use of a hypothetical scenario, the failure to adjust the benefits of hospitalization to Annals of Emergency Medicine 7
Chen et al
Measuring Patient Tolerance for Future Adverse Events
Each Patient's Numeric and Visual Thresholds Grouped by Numeric Threshold (N=234) 1 in 2 1 in 10
1 in 100
1 in 1,000
1 in 10,000 Lowest (n=115)
Patients
Intermediate (n=97)
Highest (n=22)
Figure 6. Change in patient risk threshold between numeric and visual formats. Each patient is represented by a vertical line that extends from his numeric risk threshold to his visual risk threshold. If the 2 are the same, the patient is represented by a dot. Patients are organized by their numeric values, starting with the 115 patients (in blue) who desired hospitalization at the lowest risk offered (1 in 10,000). The next group (red) is composed of the 97 patients who chose an intermediate numeric value, further sorted by whether the visual threshold was higher (n¼40), unchanged (n¼20), or lower (n¼37) than the numeric threshold. The final group (black) is 22 patients who wanted discharge even at the highest numeric risk offered (1 in 2). Readers should consider whether the change from numeric to visual was small (representing fine tuning of one’s opinion) or spans several orders of magnitude (suggesting that the 2 methods produced significantly different opinions).
each patient’s clinical circumstance, and the assumption that hospitalization would reduce morbidity by 10%. We included explicit mention of financial and opportunity costs to hospitalization in the risk tolerance script but did not include situational factors such as living alone or barriers to transportation to and from the hospital for outpatient evaluation or recurrent symptoms. We decided to represent probabilities as percentages, natural frequencies, and a visual array and to present them separately so we could study differences among them. Different results might be obtained if other methods were used to represent risk and if representations of risk had been presented simultaneously. Finally, our decision to separately examine patients who chose extreme values and those who chose intermediate ones is predicated on the assumption that those who chose extreme values did so because they were unwilling to engage in the exercise as opposed to because these extreme values represented their belief. We may also have introduced bias into the study by always having the visual risk come after the numeric risk presentation because fatigue, unwillingness to abandon one’s initial commitment to a number, or other factors may have 8 Annals of Emergency Medicine
contaminated choices made with the visual scale. It may also be that risk preferences are inherently not stable. Some naturalistic decisionmaking models emphasize social context, social roles, and role-appropriate behaviors as drivers of decisionmaking.22,23 Other sociological models of risk describe how worldviews and cultural and social institutions play a strong role in generating normative frameworks of risk, such as with conceptions of risk about nuclear energy, strongly implying that risk measurement is unstable and heterogeneous.24 Finally, psychological factors such as trust in the treating physician and in health care institutions, and health beliefs about disease course and cure that affect feelings of self-efficacy may affect risk preferences.25,26 Additionally, research assistants may have consciously or unconsciously influenced the respondents. The students did not wear white coats or other hospital uniforms and introduced themselves as student research assistants as opposed to clinicians. Although we attempted to reduce the effect of deference to authority figures by having student research assistants rather than physician researchers perform the semistructured interviews, we acknowledge that novices (ie, the patients) often defer to authority figures (ie, the medical system) for decisionmaking. Selection bias represents the greatest threat to external validity. We conducted this study at a VA hospital, which has a patient population that is different from that of the general ED community in demographics and case mix. In particular, the total number of female participants was very low. Given differences between men and women about risk perception and risk taking, making generalizations beyond male patients’ risk preferences is problematic.27,28 We chose the VA setting to minimize the role of financial and other access considerations. It is possible that some veterans have a bias toward resource use because they may view health care as a benefit already earned. This bias would lead to strong preferences for hospitalization even at very low levels of risk. We conducted the study at convenient times but minimized selection bias by including nearly all eligible patients during those times. We have no reason to believe that patients treated on weekends, night shifts, and summer months, the times underrepresented in our study, should be different from those we captured.
DISCUSSION The main conclusion from our study is that using an expected utility model to measure patients’ risk thresholds does not seem to work, either to find a stable risk preference within individuals or in groups. Although theoretically there may be an elusive combination of the right words and pictures to minimize noise and bias introduced by assessment tools, it is doubtful, given the heterogeneity found within our study and several other studies discussed below. The majority of our patients desired hospitalization when it reduced risk from 1 in 10,000 to 1 in 11,000, an implied number needed to treat to benefit of 110,000. Other studies with different designs and measurement techniques reported higher risk thresholds. A similar study that Volume
-,
no.
-
:
-
2014
Chen et al
Measuring Patient Tolerance for Future Adverse Events
used the same risk tolerance script but initially offered patients intermediate risk values expressed as percentages found that patients had a median risk threshold of 1 in 15.10 Davis et al11 demonstrated that the majority of ED patients with chest pain who were offered a hypothetical scenario in which hospitalization decreased the risk of complications of myocardial infarction from 1 in 100 to 1 in 500 preferred outpatient management (number needed to treat to benefit
