Journal of Health Communication International Perspectives
ISSN: 1081-0730 (Print) 1087-0415 (Online) Journal homepage: http://www.tandfonline.com/loi/uhcm20
Effects of Numerical Versus Foreground-Only Icon Displays on Understanding of Risk Magnitudes Eric R. Stone, Alexis R. Gabard, Aislinn E. Groves & Isaac M. Lipkus To cite this article: Eric R. Stone, Alexis R. Gabard, Aislinn E. Groves & Isaac M. Lipkus (2015) Effects of Numerical Versus Foreground-Only Icon Displays on Understanding of Risk Magnitudes, Journal of Health Communication, 20:10, 1230-1241, DOI: 10.1080/10810730.2015.1018594 To link to this article: http://dx.doi.org/10.1080/10810730.2015.1018594
Published online: 11 Jun 2015.
Submit your article to this journal
Article views: 96
View related articles
View Crossmark data
Full Terms & Conditions of access and use can be found at http://www.tandfonline.com/action/journalInformation?journalCode=uhcm20 Download by: [New York University]
Date: 03 March 2016, At: 22:09
Journal of Health Communication, 20:1230–1241, 2015 Copyright # Taylor & Francis Group, LLC ISSN: 1081-0730 print/1087-0415 online DOI: 10.1080/10810730.2015.1018594
Effects of Numerical Versus Foreground-Only Icon Displays on Understanding of Risk Magnitudes ERIC R. STONE1, ALEXIS R. GABARD1, AISLINN E. GROVES1, and ISAAC M. LIPKUS2 1
Downloaded by [New York University] at 22:09 03 March 2016
2
Department of Psychology, Wake Forest University, Winston-Salem, North Carolina, USA School of Nursing, Duke University, Durham, North Carolina, USA
The aim of this work is to advance knowledge of how to measure gist and verbatim understanding of risk magnitude information and to apply this knowledge to address whether graphics that focus on the number of people affected (the numerator of the risk ratio, i.e., the foreground) are effective displays for increasing (a) understanding of absolute and relative risk magnitudes and (b) risk avoidance. In 2 experiments, the authors examined the effects of a graphical display that used icons to represent the foreground information on measures of understanding (Experiments 1 and 2) and on perceived risk, affect, and risk aversion (Experiment 2). Consistent with prior findings, this foreground-only graphical display increased perceived risk and risk aversion; however, it also led to decreased understanding of absolute (although not relative) risk magnitudes. Methodologically, this work shows the importance of distinguishing understanding of absolute risk from understanding of relative risk magnitudes, and the need to assess gist knowledge of both types of risk. Substantively, this work shows that although using foreground-only graphical displays is an appealing risk communication strategy to increase risk aversion, doing so comes at the cost of decreased understanding of absolute risk magnitudes.
A critical public health challenge is how best to convey quantitative health information to the public (see Edwards et al., 2000; Fagerlin, Zikmund-Fisher, & Ubel, 2011; Feldman-Stewart, Kocovski, McConnell, Brundage, & MacKillip, 2000; Schapira, Nattinger, & McHorney, 2001). This article focuses on one element of this enterprise, conveying the likelihood of a risky event (e.g., probability of contracting a disease), that is, the risk magnitude. Conveying risk magnitudes numerically is challenging for multiple reasons, including people’s difficulty understanding and using low probabilities (e.g., Camerer & Kunreuther, 1989; Fisher, McClelland, & Schulze, 1989; Halpern, Blackman, & Salzman, 1989; Lipkus, 2007; Stone, Yates, & Parker, 1994), and issues of innumeracy (Fagerlin, Ubel, Smith, & Zikmund-Fisher, 2007; Peters et al., 2006). To overcome both obstacles, one frequent suggestion is to use graphical displays (for reviews, see Ancker, Senathirajah, Kukafka, & Starren, 2006; Garcia-Retamero & Cokely, 2013; Hildon, Allwood, & Black, 2012; Lipkus, 2007; Lipkus & Hollands, 1999; Spiegelhalter, Pearson, & Short, 2011; Visschers, Meertens, Passchier, & de Vries, 2009). Yet, despite considerable research regarding how to convey risk magnitudes, there exist ‘‘few best practices’’ (Lipkus, 2007, p. 709; see also Trevena et al., 2013). Alexis R. Gabard is currently affiliated with Keranetics, LLC, Winston-Salem, North Carolina, USA. Aislinn E. Groves is currently affiliated with the School for Social Work, Smith College, Northampton, Massachusetts, USA.
Address correspondence to Eric R. Stone, Department of Psychology, Wake Forest University, Box 7778 Reynolda Station, Winston-Salem, NC 27109, USA. E-mail: [email protected]
One reason for the lack of best practices is that graphical displays vary in efficacy for accomplishing different risk communication goals (Fagerlin & Peters, 2012; Lipkus, 2007; Lipkus & Hollands, 1999; Shah, Freedman, & Vekiri, 2005; ZikmundFisher, 2013). Although many risk-communication goals exist (see, e.g., Keeney & von Winterfeldt, 1986; Lipkus, 2007; Rohrmann, 1992), two common goals are to (a) increase understanding of risk magnitudes to facilitate informed decision making; and (b) modify behavior, typically to induce people to be more risk averse (such as in antismoking campaigns). One approach to increase understanding of risk magnitudes is to use displays that depict graphically both the foreground (i.e., the number of people afflicted) and the background (i.e., the population at risk). These displays depict visually the part-to-whole ratio of the risk magnitude (Ancker et al., 2006), thereby making the part-to-whole relationship ‘‘pop out’’ (Reyna & Brainerd, 2008, p. 103; see also Reyna, 2008). Many studies have shown that displays of this nature, such as stacked bar graphs or icon arrays, increase understanding of the risk-magnitude information (e.g., Fagerlin, Wang, & Ubel, 2005; Galesic, GarciaRetamero, & Gigerenzer, 2009; Garcia-Retamero & Galesic, 2010; Garcia-Retamero, Galesic, & Gigerenzer, 2010; Hawley et al., 2008; Hess, Visschers, & Siegrist, 2011; MacDonald Gibson, Rowe, Stone, & Bruine de Bruin, 2013; Okan, Garcia-Retamero, Cokely, & Maldonado, 2012; Tait, VoepelLewis, Zikmund-Fisher, & Fagerlin, 2010; Zikmund-Fisher, Ubel, et al., 2008; but see Ruiz et al., 2013). Despite the effectiveness of part-to-whole graphical displays for increasing understanding of risk magnitudes, these displays are not effective for increasing and may
1231
Understanding of Risk Magnitudes actually decrease risk aversion (Hu, Jiang, Xie, Ma, & Xu, 2014; Stone, Bruine de Bruin, Rogers, Boker, & MacDonald Gibson, under review; Stone et al., 2003). For example, in Stone and colleagues’ (2003) research, participants who reviewed displays that depicted the part-to-whole graphically judged the risk magnitude as smaller and were less risk averse compared to participants who only saw the information numerically. Thus, although displays that depict the part-towhole relationship graphically often increase understanding, they may also decrease risk aversion, an undesirable consequence if one wants to increase risk-averse behavior.
Downloaded by [New York University] at 22:09 03 March 2016
Foreground-Only Graphical Displays An alternative communication approach is to use a display that depicts only the foreground information graphically. In addition to using displays that depicted the part-to-whole graphically, Stone and colleagues (2003) included two displays that depicted the foreground graphically (using icons or bar graphs) but provided background information numerically. Participants provided with these displays judged the risk as being greater and were more risk averse in comparison to participants provided with only numerical information or displays that depicted the part-to-whole graphically. The extant literature is consistent with this conclusion: foreground-only graphical displays are effective for increasing risk aversion, at least for low-probability risk magnitudes (Chua, Yates, & Shah, 2006; Hu et al., 2014; Schirillo & Stone, 2005; Stone et al., 2003; Stone, Yates, & Parker, 1997). A key question is what effect foreground-only graphical displays have on understanding of the risk magnitudes. One concern is that depicting only the foreground graphically may call attention away from the background (Reyna, 2008; Reyna & Brainerd, 2008; Stone et al., 2003). Thus, when the risk magnitudes are small, this disproportionate attention to the foreground (number afflicted) in relation to the background (total at risk) could produce an overestimation of the risk magnitudes. If true, these displays may increase risk aversion at the cost of reduced understanding (i.e., overestimating the risk). A small amount of research has investigated this issue (Gaissmaier et al., 2012; Garcia-Retamero & Galesic, 2010; Waters, Weinstein, Colditz, & Emmons, 2006). The primary finding is that foreground-only graphical displays increase understanding, although not by as much as do part-to-whole graphical displays (Garcia-Retamero & Galesic, 2010).1 For 1 This conclusion may be limited to those with high graphical literacy. Garcia-Retamero and Galesic (2010) found improvements in understanding with foreground-only graphical displays for participants with both low and high graphical literacy, but that the improvements were greater for those with high graphical literacy. Similar to Garcia-Retamero and Galesic (2010), Gaissmaier and colleagues (2012) also included foreground-only and foregroundþbackground graphical displays as well as numerical displays. Although because of their research interests, Gaissmaier and colleagues (2012) did not test for display differences depending on whether the background was included in the graphical display, an examination of their Figure 2 shows that the foreground-only graphical display increased understanding in relation to a purely numerical display for high graphical literacy participants but decreased understanding for low graphical literacy participants.
example, in a study by Waters and colleagues (2006), participants were given information about a drug that decreased the chance of Cancer A (from 40 to 10 out of 100) but increased the chance of Cancer B (from 4 to 12 out of 100). Participants given a bar graph depicting only the foreground information correctly answered that their total cancer risk would decrease more often than did participants given only the numbers. Note, however, that to correctly answer this question, the relevant information is that the drug reduces the risk of Cancer A from 40 to 10 and increases the risk of Cancer B from 3 to 12, leading to overall decreased risk (10 þ 12 < 40 þ 3). Whether these numbers are out of 100, 1,000, or 1,000,000 is irrelevant, as long as the denominator is the same for both cancers. Thus, if background information is ignored (or downweighted), this will not influence understanding based on the measures used in Waters and colleagues’ (2006) or Garcia-Retamero and Galesic’s (2010) studies. To evaluate fully what effects foreground-only graphical displays have on understanding, it is necessary to use a broader set of understanding measures. Types of Understanding Understanding of risk-magnitude information can be evaluated in multiple ways (Cuite, Weinstein, Emmons, & Colditz, 2008; Weinstein, 1999). One distinction is between perceiving the absolute risk magnitude and the relative magnitudes of different risks (Reyna, 2008; Weinstein, 1999; see also FeldmanStewart & Brundage, 2004; Fischhoff & MacGregor, 1983; Rothman, Klein, & Weinstein, 1996). Understanding of absolute risk refers to correctly gauging the size of the risk magnitude (i.e., how large or small it is). Measuring understanding of the absolute risk is often done by asking respondents to recall (i.e., reflect back) the risk magnitude (e.g., Miron-Shatz, Hanoch, Graef, & Sagi, 2009; Tait et al., 2010; Woloshin, Schwartz, Black, & Welch, 1999; Zikmund-Fisher, Fagerlin, & Ubel, 2008; Zikmund-Fisher et al., 2014). When two or more risky events exist, understanding of relative magnitudes can be assessed as well.2 Understanding of relative risk is typically evaluated by asking participants to state which of two risks is greater or to rank a set of risk magnitudes (e.g., Diefenbach, Weinstein, & O’Reilly, 1993; Hawley et al., 2008; Sheridan, Pignone, & Lewis, 2003; Timmermans, Molewijk, Stiggelbout, & Kievit, 2004; Waters, Weinstein, Colditz, & Emmons, 2006, 2007a, 2007b). Importantly, one can know the relative ordering of risk magnitudes without knowing the size of the absolute risk magnitudes, for example, by knowing that more people die from heart disease than from stroke but underestimating both risk magnitudes (see Lichtenstein, Slovic, Fischhoff, Layman, & Combs, 1978). Alternatively, one could have a
2 Note that we are using the term relative risk to indicate the relative magnitude of two or more risky events to one particular person. Others (e.g., Woloshin et al., 1999) have distinguished one person’s absolute risk from that person’s relative risk in comparison with other people regarding the particular risky event. Although both types of relative risk are important, the scenarios we examine assume that we only have general risk statistics, not ones tailored to a particular individual. Thus, the present article focuses solely on the first type of relative risk.
Downloaded by [New York University] at 22:09 03 March 2016
1232 general understanding of how large or small a set of risk magnitudes is without knowing their ordering. A second distinction is between verbatim and gist understanding. According to fuzzy trace theory (e.g., Reyna, 2008; Reyna & Brainerd, 2008), people encode both verbatim (literal) information as well as its gist (meaning). One common approach to measure understanding is to ask people to recall the numbers provided in the risk communication (i.e., the literal, verbatim information). However, this approach has been criticized because accurate recall does not ensure that the information can be used effectively. Rather, decisions are typically made based on gist (see, e.g., Reyna, 2008; Timmermans et al., 2004; Weinstein, 1999; Wilhelms & Reyna, 2013; Wolfe, 2006; Zikmund-Fisher, 2013). Thus, it is important to include measures of gist along with measures of verbatim understanding (e.g., Feldman-Stewart, Brundage, & Zotov, 2007; Hawley et al., 2008; Wolfe, 2006). Although gist understanding is frequently captured by asking people to provide the relative ordering of two or more risks (e.g., Feldman-Stewart et al., 2007; Gaissmaier et al., 2012; McCaffery et al., 2012; Ruiz et al., 2013; Tait et al., 2010; Zikmund-Fisher, Ubel, et al., 2008; for exceptions, see Brewer, Gilkey, Lillie, Hesse, & Sheridan, 2012; Brewer, Richman, DeFrank, Reyna, & Carey, 2012), it is important to recognize that this is only one type of gist. As discussed by Reyna (2008), one can misunderstand the gist of the absolute risk (i.e., by thinking that the risks are much larger or smaller than they really are) or misunderstand the gist of the relative risk (i.e., by not understanding which of two risk magnitudes is larger). Equating gist with ‘‘relative ordering’’—as has often been done in the literature—misses that a display can work well for capturing the gist of the relative risk but be poor at conveying the gist of the absolute magnitudes. Indeed, foregroundonly graphical displays may be effective at conveying the gist of the relative risk, but be poor at conveying the gist of the absolute magnitudes, because they call attention away from the background (Reyna, 2008).
E. R. Stone et al. Experiment 1 We created risk scenarios about five infectious diseases. The likelihood magnitude information was conveyed either numerically as frequency information or by a foregroundonly graphical display. In addition, we varied whether the risk ratios for the five diseases included a common or a noncommon denominator to determine whether any graphical effect would depend on using a common or noncommon denominator (see Garcia-Retamero et al., 2010; Okan et al., 2012).
Method Participants Three hundred and fourteen students (54.1% female) enrolled in an introductory psychology course participated as one option for partially fulfilling a course requirement. Most of the sample (89.2%) had no prior experience with the diseases presented. Experimental Stimuli Disease Packet The disease packets described five real infectious diseases by providing their symptoms, length, mode of transmission, and prevalence among the general American public. All were of moderately low probability (i.e., incidence rates from .002 to .084) and disease prevalence was estimated using data from the Centers for Disease Control and Prevention (CDC). The diseases and associated prevalence rates were as follows: adenovirus (66 in 1,000), brucellosis (9 in 1,000), crypto (2 in 1,000), OPC (30 in 1,000), and parvovirus B19 (84 in 1,000). Summary Sheet We constructed a summary sheet of the five diseases, which contained an abbreviated description of each disease (e.g., symptoms) but did not include prevalence information.
The Present Research The present research was designed to address two issues, a methodological and a substantive one. Methodologically, we aimed to increase our knowledge of measures of understanding of risk-magnitude information. Concurrently, we tested the substantive issue of whether foreground-only graphical displays increase or decrease understanding of risk magnitude information in relation to a purely numerical display. Our second experiment also included measures of perceived likelihood, negative affect, and risk aversion to examine the effect of foreground-only graphical displays across a broad set of outcome variables. In addition, we examined whether any effects would be moderated by participants’ numeracy levels, as decisions and understanding are influenced by numerical ability (see Hanoch, Miron-Shatz, & Himmelstein, 2010; Lipkus, Samsa, & Rimer, 2001; Peters et al., 2006; Reyna, Nelson, Han, & Dieckmann, 2009) and numeracy skills may moderate the effects of different risk communication formats (see Fagerlin et al., 2007; GarciaRetamero & Cokely, 2013; Spiegelhalter et al., 2011).
Measures Recall Memory For each disease, participants filled in ‘‘_____ per _____ people’’ would contract that disease in their lifetime. Because these questions were open-ended and the chance of occurrence for each disease was small, scoring correctness as the deviation from the correct answer has the potential to produce large positive skew (see Miron-Shatz et al., 2009, for a discussion of this issue). Thus, for each question, we scored correctness on a 3-point scale (correct, close to correct, far from correct) to account for the positive skew (see Stone et al., under review, for more details). Each participant’s recall memory score was the sum of the points they received for each disease. Recognition Memory For each disease, participants responded to a multiple-choice question asking them to select the correct risk magnitude. For example, for brucellosis in the common denominator
1233
Downloaded by [New York University] at 22:09 03 March 2016
Understanding of Risk Magnitudes
Fig. 1. Risk magnitude displays for Brucellosis, common denominator condition, used in Experiment 1: (a) numerical condition; (b) graphical condition.
condition, the response options were ‘‘a. 30 per 1,000 people’’; ‘‘b. 4 per 1,000 people’’; and ‘‘c. 9 per 1,000 people.’’ Because the provided information was different in the noncommon denominator condition, the choice options also differed. Each participant’s recognition memory score was the percentage of questions answered correctly. Rankings Participants rank ordered the diseases from 1 to 5 (most likely to least likely to affect the typical American). Responses were correlated (using a Spearman correlation) with actual rankings. Comparing to Known Risks One approach for assessing gist is to determine whether people are able to place facts in the context of their existing knowledge by having them compare communicated risk magnitudes to other, presumably known, risks (Halpern et al., 1989; see also Slovic, Fischhoff & Lichtenstein, 1980; Weinstein, 1999). To this end, we asked participants to state whether the typical American’s risk of contracting each disease was higher or lower than that of some presumably known event. The five comparison events were as follows: dying from falling down the stairs, contracting HIV in the United States this year, being unemployed this month, dying from cardiovascular disease, and getting pregnant from having sex once while using the pill as a contraceptive. We computed the percentage correct across these questions. Perceived Understanding Several studies assessed understanding by asking participants how well they understood the information (e.g., Connelly & Knuth, 1998; Wright, Whitwell, Takeichi, Hankins, & Marteau, 2009). Consistent with that work, participants rated how well they thought they understood the disease prevalence information by circling a number from 1 (did not understand) to 5 (completely understood).
Numeracy We assessed numerical ability with the three-item numeracy scale (Schwartz, Woloshin, Black, & Welch, 1997). On average, participants did well on the numeracy questionnaire (3 correct ¼ 57.6%, 2 correct ¼ 28.0%, 1 correct ¼ 9.2%, 0 correct ¼ 5.1%). Thus, we dichotomized this measure into all correct versus not all correct.3
Procedure We used a 2 (display type: graphical vs. numerical) 2 (denominator type: common vs. noncommon) betweensubjects design. The dependent variables were the five measures of understanding, assessed in the following order: perceived understanding, rankings, recall memory, comparing to known risks, and recognition memory.4 3 This procedure of dichotomizing a skewed measure of numeracy and related constructs is consistent with the approach typically taken in the literature (e.g., Gaissmaier et al., 2012; Peters et al., 2006). However, it has the disadvantage of reducing the power of the analysis by treating 0, 1, and 2 correct as being equivalent. Thus, to provide an additional test of whether the participants’ numeracy levels moderated any of our effects, we conducted regression analyses in which we regressed each of our aggregated measures (overall understanding in Experiment 1; understanding of absolute risk, understanding of relative risk, perceived likelihood, negative affective responses, and risk aversion in Experiment 2) on the same predictor variables entered in the main ANOVAs, but with numeracy and its interactions computed on the continuous (nondichotomized) measure of numeracy. None of the interactions with display format were significant, all ps > .11. 4 We also developed an additional measure of understanding based on whether participants made decisions that were consistent with their personal values (see Carrigan, Gardner, Conner, & Maule, 2004; Edwards & Elwyn, 1999; O’Connor, 1995), which was assessed after the other five measures of understanding. The results with this measure in both studies were somewhat weaker than with most of the other measures, so we do not discuss it further here. However, we encourage other researchers to work on developing this type of measure, as high-quality decisions is arguably the best standard for determining whether the conveyed information is fully understood.
Downloaded by [New York University] at 22:09 03 March 2016
1234
E. R. Stone et al.
Participants first received the disease packet. They were told to imagine working for the CDC and that their supervisor asked them to read about ‘‘five specific emerging infectious diseases and be ready to answer questions concerning each of the diseases at a meeting in five minutes.’’ The numerical condition displayed the risk magnitudes in numerical (frequency) form; the graphical condition displayed the number of people affected graphically using stick figures and the number at risk (the background) numerically. In addition, half of the participants were given the risk magnitudes with a common denominator (i.e., all likelihood information was given out of 1,000), whereas half were given the risk magnitudes with a noncommon denominator (e.g., likelihood information for Adenovirus was given out of 500, for Brucellosis out of 5,000). Figure 1 depicts the numerical and graphical displays for the Brucellosis, common denominator condition. In addition, participants received the summary sheet, which they kept throughout the study to help them differentiate the diseases when completing the dependent measures. Participants were given 5 min to review the information about the five diseases and told not to take notes on the information. The disease information was given in alphabetical order (i.e., for adenovirus, brucellosis, crypto, OPC, parvovirus B19). However, because of an experimental error, in one of the conditions the parvovirus information was given before the OPC information. Because this could have led those participants to be at a disadvantage on some understanding items (given that the understanding questions were asked in alphabetical order), all analyses were done on just the first three diseases. After 5 min elapsed, participants turned the packet of information over, although they were allowed to keep the summary sheet face up. They then completed the measures of understanding, followed by the numeracy questionnaire.
Results and Discussion As shown in Table 1, there was reasonably good understanding demonstrated for all measures except for comparing to
known risks, where only 53.5% of the questions were answered correctly (50% was chance). We suspect the problem was with our choice of comparison items. Although we spent considerable effort identifying comparison events whose magnitude would be common knowledge, it appears we were not successful. Part of the difficulty may be our use of an undergraduate sample—participants with more experience with risky activities, such as those used by Halpern and colleagues (1989), might be better able to answer these questions. Because of considerable negative skew for the rankings measure, we winsorized the left-side of this distribution at zero, thereby treating a zero correlation or below as reflecting a lack of understanding. Consistency in Understanding Measures There were positive correlations among all measures, with the exception of comparing to known risks, which was not significantly correlated with any other measure. With this item removed, a factor analysis revealed that all of the measures loaded on one factor (eigenvalue ¼ 2.01, all other eigenvalues .48, 50.3% of the variance explained, average correlation ¼ .32, Cronbach’s alpha ¼ .66 (see Table 2). The following analyses are thus conducted on all dependent measures except for comparing to known risks. Effect of Display Type To equate the influence of each of the four remaining understanding measures, we z-standardized each measure and then conducted a 2 (display format: numerical vs. graphical) 2 (denominator consistency: common denominator vs. noncommon denominator) 2 (numeracy: low vs. high) by 4 (understanding measure) repeated-measures analysis of variance (ANOVA), where understanding measure was the repeated-measures variable. There were significant main effects of each of the between-subjects variables. Participants presented with numerical information understood the
Table 1. Average levels of the understanding measures Average response Understanding measure Recall memory (fill-in-the-blanks) Recognition memory (multiple choice) Rankings: Correlation with actual rankings2 Comparing to known risks Perceived understanding Gist recognition Gist recall (GRcl) GRcl correlated with actual likelihoods Worry correlated with actual likelihoods 1
Experiment 1
Experiment 2 1
M ¼ 3.77 (on a 0–6 scale) M ¼ 84.5% correct M correlation ¼ 0.74; median ¼ 1.0 M ¼ 53.5% correct M ¼ 4.36 (on a 1–5 scale)
M correlation ¼ 0.70; median ¼ 0.90 M ¼ 4.37 (on a 1–5 scale) M ¼ 61.2% correct M ¼ 3.65 (on a 0–10 scale)3 M correlation ¼ 0.734; median ¼ 0.92 M correlation ¼ 0.505; median ¼ 0.67
Of the responses, 53.5% were exactly right, 18.8% were close to correct, and 27.7% were not close to correct. Means are prewinsorization. After winsorization, the means increased to M ¼ 0.80 (Experiment 1) and M ¼ 0.75 (Experiment 2). 3 Of the responses, 25.5% were exactly right; 22.0% were close to correct; and 52.5% were not close to correct. 4 Mean is prewinsorization. After winsorization, the mean increased to M ¼ 0.75. 5 Mean is prewinsorization. After winsorization, the mean increased to M ¼ 0.56. 2
1235
Understanding of Risk Magnitudes Table 2. Factor loadings of the understanding measures
Understanding measure
Experiment 2 factor loadings2
Experiment 1 factor loading1
Absolute factor
Relative factor
.82 .77 .71 .04 .49
.08 .60 .81 .81 .15 .15
.88 .07 .27 .33 .88 .63
Recall memory (fill-in-the-blanks) Recognition memory (multiple choice) Rankings: Correlation with actual rankings Comparing to known risks Perceived understanding Gist recognition Gist recall (GRcl) GRcl correlated with actual likelihoods Worry correlated with actual likelihoods 1
Downloaded by [New York University] at 22:09 03 March 2016
A factor analysis that did not include comparing to known risks produced almost identical results to those shown here, with all loadings within .01 of those shown. 2 All loadings from principal components extraction, varimax rotation.
information better than did participants presented with graphical information, F(1, 297) ¼ 52.51, p < .001; participants presented with a common denominator understood the information better than did participants presented with a noncommon denominator, F(1, 297) ¼ 51.37, p < .001; and participants with higher numeracy demonstrated better understanding than did participants with lower numeracy, F(1, 297) ¼ 6.62, p ¼ .01. There were no interactions among the between-subjects variables (all ps > .29). Thus, the greater understanding produced by the numerical display held regardless of whether a common or noncommon denominator was used and regardless of the participant’s numeracy level. There was, however, a highly significant interaction between display format and understanding measure, indicating that the superiority of the numerical displays was greater for some understanding measures than for
others, multivariate F(3, 295) ¼ 7.31, p < .001. Univariate ANOVAs conducted separately on each of the understanding measures show that the numerical display produced greater understanding than did the graphical display on all of the measures except for comparing to known risks (see Table 3). However, this display effect is particularly strong for the understanding questions that ask about absolute risk. The only other significant interaction was between denominator consistency and understanding measure, multivariate F(3, 295) ¼ 9.09, p < .001. As seen in Table 4, the effect of denominator consistency was particularly strong on rankings, our measure of understanding of the relative risk. Hence, although using noncommon denominators is generally problematic (see Garcia-Retamero et al., 2010; Okan et al., 2012; Paling, 2003), this seems to cause particular difficulty with understanding relative risk magnitudes.
Table 3. Experiment 1 mean understanding levels for numerical and graphical displays Understanding measure Recall memory (fill-in-the-blanks) Recognition memory (multiple choice) Rankings: Correlation with actual rankings Comparing to known risks Perceived understanding
Numerical display
Graphical display
F
P
4.55 91.0% 0.83 54.0% 4.52
3.01 78.0% 0.76 53.1% 4.21
F(1, 306) ¼ 62.82 F(1, 304) ¼ 26.00 F(1, 304) ¼ 3.62 F(1, 303) ¼ 0.43 F(1, 301) ¼ 14.49
ISSN: 1081-0730 (Print) 1087-0415 (Online) Journal homepage: http://www.tandfonline.com/loi/uhcm20
Effects of Numerical Versus Foreground-Only Icon Displays on Understanding of Risk Magnitudes Eric R. Stone, Alexis R. Gabard, Aislinn E. Groves & Isaac M. Lipkus To cite this article: Eric R. Stone, Alexis R. Gabard, Aislinn E. Groves & Isaac M. Lipkus (2015) Effects of Numerical Versus Foreground-Only Icon Displays on Understanding of Risk Magnitudes, Journal of Health Communication, 20:10, 1230-1241, DOI: 10.1080/10810730.2015.1018594 To link to this article: http://dx.doi.org/10.1080/10810730.2015.1018594
Published online: 11 Jun 2015.
Submit your article to this journal
Article views: 96
View related articles
View Crossmark data
Full Terms & Conditions of access and use can be found at http://www.tandfonline.com/action/journalInformation?journalCode=uhcm20 Download by: [New York University]
Date: 03 March 2016, At: 22:09
Journal of Health Communication, 20:1230–1241, 2015 Copyright # Taylor & Francis Group, LLC ISSN: 1081-0730 print/1087-0415 online DOI: 10.1080/10810730.2015.1018594
Effects of Numerical Versus Foreground-Only Icon Displays on Understanding of Risk Magnitudes ERIC R. STONE1, ALEXIS R. GABARD1, AISLINN E. GROVES1, and ISAAC M. LIPKUS2 1
Downloaded by [New York University] at 22:09 03 March 2016
2
Department of Psychology, Wake Forest University, Winston-Salem, North Carolina, USA School of Nursing, Duke University, Durham, North Carolina, USA
The aim of this work is to advance knowledge of how to measure gist and verbatim understanding of risk magnitude information and to apply this knowledge to address whether graphics that focus on the number of people affected (the numerator of the risk ratio, i.e., the foreground) are effective displays for increasing (a) understanding of absolute and relative risk magnitudes and (b) risk avoidance. In 2 experiments, the authors examined the effects of a graphical display that used icons to represent the foreground information on measures of understanding (Experiments 1 and 2) and on perceived risk, affect, and risk aversion (Experiment 2). Consistent with prior findings, this foreground-only graphical display increased perceived risk and risk aversion; however, it also led to decreased understanding of absolute (although not relative) risk magnitudes. Methodologically, this work shows the importance of distinguishing understanding of absolute risk from understanding of relative risk magnitudes, and the need to assess gist knowledge of both types of risk. Substantively, this work shows that although using foreground-only graphical displays is an appealing risk communication strategy to increase risk aversion, doing so comes at the cost of decreased understanding of absolute risk magnitudes.
A critical public health challenge is how best to convey quantitative health information to the public (see Edwards et al., 2000; Fagerlin, Zikmund-Fisher, & Ubel, 2011; Feldman-Stewart, Kocovski, McConnell, Brundage, & MacKillip, 2000; Schapira, Nattinger, & McHorney, 2001). This article focuses on one element of this enterprise, conveying the likelihood of a risky event (e.g., probability of contracting a disease), that is, the risk magnitude. Conveying risk magnitudes numerically is challenging for multiple reasons, including people’s difficulty understanding and using low probabilities (e.g., Camerer & Kunreuther, 1989; Fisher, McClelland, & Schulze, 1989; Halpern, Blackman, & Salzman, 1989; Lipkus, 2007; Stone, Yates, & Parker, 1994), and issues of innumeracy (Fagerlin, Ubel, Smith, & Zikmund-Fisher, 2007; Peters et al., 2006). To overcome both obstacles, one frequent suggestion is to use graphical displays (for reviews, see Ancker, Senathirajah, Kukafka, & Starren, 2006; Garcia-Retamero & Cokely, 2013; Hildon, Allwood, & Black, 2012; Lipkus, 2007; Lipkus & Hollands, 1999; Spiegelhalter, Pearson, & Short, 2011; Visschers, Meertens, Passchier, & de Vries, 2009). Yet, despite considerable research regarding how to convey risk magnitudes, there exist ‘‘few best practices’’ (Lipkus, 2007, p. 709; see also Trevena et al., 2013). Alexis R. Gabard is currently affiliated with Keranetics, LLC, Winston-Salem, North Carolina, USA. Aislinn E. Groves is currently affiliated with the School for Social Work, Smith College, Northampton, Massachusetts, USA.
Address correspondence to Eric R. Stone, Department of Psychology, Wake Forest University, Box 7778 Reynolda Station, Winston-Salem, NC 27109, USA. E-mail: [email protected]
One reason for the lack of best practices is that graphical displays vary in efficacy for accomplishing different risk communication goals (Fagerlin & Peters, 2012; Lipkus, 2007; Lipkus & Hollands, 1999; Shah, Freedman, & Vekiri, 2005; ZikmundFisher, 2013). Although many risk-communication goals exist (see, e.g., Keeney & von Winterfeldt, 1986; Lipkus, 2007; Rohrmann, 1992), two common goals are to (a) increase understanding of risk magnitudes to facilitate informed decision making; and (b) modify behavior, typically to induce people to be more risk averse (such as in antismoking campaigns). One approach to increase understanding of risk magnitudes is to use displays that depict graphically both the foreground (i.e., the number of people afflicted) and the background (i.e., the population at risk). These displays depict visually the part-to-whole ratio of the risk magnitude (Ancker et al., 2006), thereby making the part-to-whole relationship ‘‘pop out’’ (Reyna & Brainerd, 2008, p. 103; see also Reyna, 2008). Many studies have shown that displays of this nature, such as stacked bar graphs or icon arrays, increase understanding of the risk-magnitude information (e.g., Fagerlin, Wang, & Ubel, 2005; Galesic, GarciaRetamero, & Gigerenzer, 2009; Garcia-Retamero & Galesic, 2010; Garcia-Retamero, Galesic, & Gigerenzer, 2010; Hawley et al., 2008; Hess, Visschers, & Siegrist, 2011; MacDonald Gibson, Rowe, Stone, & Bruine de Bruin, 2013; Okan, Garcia-Retamero, Cokely, & Maldonado, 2012; Tait, VoepelLewis, Zikmund-Fisher, & Fagerlin, 2010; Zikmund-Fisher, Ubel, et al., 2008; but see Ruiz et al., 2013). Despite the effectiveness of part-to-whole graphical displays for increasing understanding of risk magnitudes, these displays are not effective for increasing and may
1231
Understanding of Risk Magnitudes actually decrease risk aversion (Hu, Jiang, Xie, Ma, & Xu, 2014; Stone, Bruine de Bruin, Rogers, Boker, & MacDonald Gibson, under review; Stone et al., 2003). For example, in Stone and colleagues’ (2003) research, participants who reviewed displays that depicted the part-to-whole graphically judged the risk magnitude as smaller and were less risk averse compared to participants who only saw the information numerically. Thus, although displays that depict the part-towhole relationship graphically often increase understanding, they may also decrease risk aversion, an undesirable consequence if one wants to increase risk-averse behavior.
Downloaded by [New York University] at 22:09 03 March 2016
Foreground-Only Graphical Displays An alternative communication approach is to use a display that depicts only the foreground information graphically. In addition to using displays that depicted the part-to-whole graphically, Stone and colleagues (2003) included two displays that depicted the foreground graphically (using icons or bar graphs) but provided background information numerically. Participants provided with these displays judged the risk as being greater and were more risk averse in comparison to participants provided with only numerical information or displays that depicted the part-to-whole graphically. The extant literature is consistent with this conclusion: foreground-only graphical displays are effective for increasing risk aversion, at least for low-probability risk magnitudes (Chua, Yates, & Shah, 2006; Hu et al., 2014; Schirillo & Stone, 2005; Stone et al., 2003; Stone, Yates, & Parker, 1997). A key question is what effect foreground-only graphical displays have on understanding of the risk magnitudes. One concern is that depicting only the foreground graphically may call attention away from the background (Reyna, 2008; Reyna & Brainerd, 2008; Stone et al., 2003). Thus, when the risk magnitudes are small, this disproportionate attention to the foreground (number afflicted) in relation to the background (total at risk) could produce an overestimation of the risk magnitudes. If true, these displays may increase risk aversion at the cost of reduced understanding (i.e., overestimating the risk). A small amount of research has investigated this issue (Gaissmaier et al., 2012; Garcia-Retamero & Galesic, 2010; Waters, Weinstein, Colditz, & Emmons, 2006). The primary finding is that foreground-only graphical displays increase understanding, although not by as much as do part-to-whole graphical displays (Garcia-Retamero & Galesic, 2010).1 For 1 This conclusion may be limited to those with high graphical literacy. Garcia-Retamero and Galesic (2010) found improvements in understanding with foreground-only graphical displays for participants with both low and high graphical literacy, but that the improvements were greater for those with high graphical literacy. Similar to Garcia-Retamero and Galesic (2010), Gaissmaier and colleagues (2012) also included foreground-only and foregroundþbackground graphical displays as well as numerical displays. Although because of their research interests, Gaissmaier and colleagues (2012) did not test for display differences depending on whether the background was included in the graphical display, an examination of their Figure 2 shows that the foreground-only graphical display increased understanding in relation to a purely numerical display for high graphical literacy participants but decreased understanding for low graphical literacy participants.
example, in a study by Waters and colleagues (2006), participants were given information about a drug that decreased the chance of Cancer A (from 40 to 10 out of 100) but increased the chance of Cancer B (from 4 to 12 out of 100). Participants given a bar graph depicting only the foreground information correctly answered that their total cancer risk would decrease more often than did participants given only the numbers. Note, however, that to correctly answer this question, the relevant information is that the drug reduces the risk of Cancer A from 40 to 10 and increases the risk of Cancer B from 3 to 12, leading to overall decreased risk (10 þ 12 < 40 þ 3). Whether these numbers are out of 100, 1,000, or 1,000,000 is irrelevant, as long as the denominator is the same for both cancers. Thus, if background information is ignored (or downweighted), this will not influence understanding based on the measures used in Waters and colleagues’ (2006) or Garcia-Retamero and Galesic’s (2010) studies. To evaluate fully what effects foreground-only graphical displays have on understanding, it is necessary to use a broader set of understanding measures. Types of Understanding Understanding of risk-magnitude information can be evaluated in multiple ways (Cuite, Weinstein, Emmons, & Colditz, 2008; Weinstein, 1999). One distinction is between perceiving the absolute risk magnitude and the relative magnitudes of different risks (Reyna, 2008; Weinstein, 1999; see also FeldmanStewart & Brundage, 2004; Fischhoff & MacGregor, 1983; Rothman, Klein, & Weinstein, 1996). Understanding of absolute risk refers to correctly gauging the size of the risk magnitude (i.e., how large or small it is). Measuring understanding of the absolute risk is often done by asking respondents to recall (i.e., reflect back) the risk magnitude (e.g., Miron-Shatz, Hanoch, Graef, & Sagi, 2009; Tait et al., 2010; Woloshin, Schwartz, Black, & Welch, 1999; Zikmund-Fisher, Fagerlin, & Ubel, 2008; Zikmund-Fisher et al., 2014). When two or more risky events exist, understanding of relative magnitudes can be assessed as well.2 Understanding of relative risk is typically evaluated by asking participants to state which of two risks is greater or to rank a set of risk magnitudes (e.g., Diefenbach, Weinstein, & O’Reilly, 1993; Hawley et al., 2008; Sheridan, Pignone, & Lewis, 2003; Timmermans, Molewijk, Stiggelbout, & Kievit, 2004; Waters, Weinstein, Colditz, & Emmons, 2006, 2007a, 2007b). Importantly, one can know the relative ordering of risk magnitudes without knowing the size of the absolute risk magnitudes, for example, by knowing that more people die from heart disease than from stroke but underestimating both risk magnitudes (see Lichtenstein, Slovic, Fischhoff, Layman, & Combs, 1978). Alternatively, one could have a
2 Note that we are using the term relative risk to indicate the relative magnitude of two or more risky events to one particular person. Others (e.g., Woloshin et al., 1999) have distinguished one person’s absolute risk from that person’s relative risk in comparison with other people regarding the particular risky event. Although both types of relative risk are important, the scenarios we examine assume that we only have general risk statistics, not ones tailored to a particular individual. Thus, the present article focuses solely on the first type of relative risk.
Downloaded by [New York University] at 22:09 03 March 2016
1232 general understanding of how large or small a set of risk magnitudes is without knowing their ordering. A second distinction is between verbatim and gist understanding. According to fuzzy trace theory (e.g., Reyna, 2008; Reyna & Brainerd, 2008), people encode both verbatim (literal) information as well as its gist (meaning). One common approach to measure understanding is to ask people to recall the numbers provided in the risk communication (i.e., the literal, verbatim information). However, this approach has been criticized because accurate recall does not ensure that the information can be used effectively. Rather, decisions are typically made based on gist (see, e.g., Reyna, 2008; Timmermans et al., 2004; Weinstein, 1999; Wilhelms & Reyna, 2013; Wolfe, 2006; Zikmund-Fisher, 2013). Thus, it is important to include measures of gist along with measures of verbatim understanding (e.g., Feldman-Stewart, Brundage, & Zotov, 2007; Hawley et al., 2008; Wolfe, 2006). Although gist understanding is frequently captured by asking people to provide the relative ordering of two or more risks (e.g., Feldman-Stewart et al., 2007; Gaissmaier et al., 2012; McCaffery et al., 2012; Ruiz et al., 2013; Tait et al., 2010; Zikmund-Fisher, Ubel, et al., 2008; for exceptions, see Brewer, Gilkey, Lillie, Hesse, & Sheridan, 2012; Brewer, Richman, DeFrank, Reyna, & Carey, 2012), it is important to recognize that this is only one type of gist. As discussed by Reyna (2008), one can misunderstand the gist of the absolute risk (i.e., by thinking that the risks are much larger or smaller than they really are) or misunderstand the gist of the relative risk (i.e., by not understanding which of two risk magnitudes is larger). Equating gist with ‘‘relative ordering’’—as has often been done in the literature—misses that a display can work well for capturing the gist of the relative risk but be poor at conveying the gist of the absolute magnitudes. Indeed, foregroundonly graphical displays may be effective at conveying the gist of the relative risk, but be poor at conveying the gist of the absolute magnitudes, because they call attention away from the background (Reyna, 2008).
E. R. Stone et al. Experiment 1 We created risk scenarios about five infectious diseases. The likelihood magnitude information was conveyed either numerically as frequency information or by a foregroundonly graphical display. In addition, we varied whether the risk ratios for the five diseases included a common or a noncommon denominator to determine whether any graphical effect would depend on using a common or noncommon denominator (see Garcia-Retamero et al., 2010; Okan et al., 2012).
Method Participants Three hundred and fourteen students (54.1% female) enrolled in an introductory psychology course participated as one option for partially fulfilling a course requirement. Most of the sample (89.2%) had no prior experience with the diseases presented. Experimental Stimuli Disease Packet The disease packets described five real infectious diseases by providing their symptoms, length, mode of transmission, and prevalence among the general American public. All were of moderately low probability (i.e., incidence rates from .002 to .084) and disease prevalence was estimated using data from the Centers for Disease Control and Prevention (CDC). The diseases and associated prevalence rates were as follows: adenovirus (66 in 1,000), brucellosis (9 in 1,000), crypto (2 in 1,000), OPC (30 in 1,000), and parvovirus B19 (84 in 1,000). Summary Sheet We constructed a summary sheet of the five diseases, which contained an abbreviated description of each disease (e.g., symptoms) but did not include prevalence information.
The Present Research The present research was designed to address two issues, a methodological and a substantive one. Methodologically, we aimed to increase our knowledge of measures of understanding of risk-magnitude information. Concurrently, we tested the substantive issue of whether foreground-only graphical displays increase or decrease understanding of risk magnitude information in relation to a purely numerical display. Our second experiment also included measures of perceived likelihood, negative affect, and risk aversion to examine the effect of foreground-only graphical displays across a broad set of outcome variables. In addition, we examined whether any effects would be moderated by participants’ numeracy levels, as decisions and understanding are influenced by numerical ability (see Hanoch, Miron-Shatz, & Himmelstein, 2010; Lipkus, Samsa, & Rimer, 2001; Peters et al., 2006; Reyna, Nelson, Han, & Dieckmann, 2009) and numeracy skills may moderate the effects of different risk communication formats (see Fagerlin et al., 2007; GarciaRetamero & Cokely, 2013; Spiegelhalter et al., 2011).
Measures Recall Memory For each disease, participants filled in ‘‘_____ per _____ people’’ would contract that disease in their lifetime. Because these questions were open-ended and the chance of occurrence for each disease was small, scoring correctness as the deviation from the correct answer has the potential to produce large positive skew (see Miron-Shatz et al., 2009, for a discussion of this issue). Thus, for each question, we scored correctness on a 3-point scale (correct, close to correct, far from correct) to account for the positive skew (see Stone et al., under review, for more details). Each participant’s recall memory score was the sum of the points they received for each disease. Recognition Memory For each disease, participants responded to a multiple-choice question asking them to select the correct risk magnitude. For example, for brucellosis in the common denominator
1233
Downloaded by [New York University] at 22:09 03 March 2016
Understanding of Risk Magnitudes
Fig. 1. Risk magnitude displays for Brucellosis, common denominator condition, used in Experiment 1: (a) numerical condition; (b) graphical condition.
condition, the response options were ‘‘a. 30 per 1,000 people’’; ‘‘b. 4 per 1,000 people’’; and ‘‘c. 9 per 1,000 people.’’ Because the provided information was different in the noncommon denominator condition, the choice options also differed. Each participant’s recognition memory score was the percentage of questions answered correctly. Rankings Participants rank ordered the diseases from 1 to 5 (most likely to least likely to affect the typical American). Responses were correlated (using a Spearman correlation) with actual rankings. Comparing to Known Risks One approach for assessing gist is to determine whether people are able to place facts in the context of their existing knowledge by having them compare communicated risk magnitudes to other, presumably known, risks (Halpern et al., 1989; see also Slovic, Fischhoff & Lichtenstein, 1980; Weinstein, 1999). To this end, we asked participants to state whether the typical American’s risk of contracting each disease was higher or lower than that of some presumably known event. The five comparison events were as follows: dying from falling down the stairs, contracting HIV in the United States this year, being unemployed this month, dying from cardiovascular disease, and getting pregnant from having sex once while using the pill as a contraceptive. We computed the percentage correct across these questions. Perceived Understanding Several studies assessed understanding by asking participants how well they understood the information (e.g., Connelly & Knuth, 1998; Wright, Whitwell, Takeichi, Hankins, & Marteau, 2009). Consistent with that work, participants rated how well they thought they understood the disease prevalence information by circling a number from 1 (did not understand) to 5 (completely understood).
Numeracy We assessed numerical ability with the three-item numeracy scale (Schwartz, Woloshin, Black, & Welch, 1997). On average, participants did well on the numeracy questionnaire (3 correct ¼ 57.6%, 2 correct ¼ 28.0%, 1 correct ¼ 9.2%, 0 correct ¼ 5.1%). Thus, we dichotomized this measure into all correct versus not all correct.3
Procedure We used a 2 (display type: graphical vs. numerical) 2 (denominator type: common vs. noncommon) betweensubjects design. The dependent variables were the five measures of understanding, assessed in the following order: perceived understanding, rankings, recall memory, comparing to known risks, and recognition memory.4 3 This procedure of dichotomizing a skewed measure of numeracy and related constructs is consistent with the approach typically taken in the literature (e.g., Gaissmaier et al., 2012; Peters et al., 2006). However, it has the disadvantage of reducing the power of the analysis by treating 0, 1, and 2 correct as being equivalent. Thus, to provide an additional test of whether the participants’ numeracy levels moderated any of our effects, we conducted regression analyses in which we regressed each of our aggregated measures (overall understanding in Experiment 1; understanding of absolute risk, understanding of relative risk, perceived likelihood, negative affective responses, and risk aversion in Experiment 2) on the same predictor variables entered in the main ANOVAs, but with numeracy and its interactions computed on the continuous (nondichotomized) measure of numeracy. None of the interactions with display format were significant, all ps > .11. 4 We also developed an additional measure of understanding based on whether participants made decisions that were consistent with their personal values (see Carrigan, Gardner, Conner, & Maule, 2004; Edwards & Elwyn, 1999; O’Connor, 1995), which was assessed after the other five measures of understanding. The results with this measure in both studies were somewhat weaker than with most of the other measures, so we do not discuss it further here. However, we encourage other researchers to work on developing this type of measure, as high-quality decisions is arguably the best standard for determining whether the conveyed information is fully understood.
Downloaded by [New York University] at 22:09 03 March 2016
1234
E. R. Stone et al.
Participants first received the disease packet. They were told to imagine working for the CDC and that their supervisor asked them to read about ‘‘five specific emerging infectious diseases and be ready to answer questions concerning each of the diseases at a meeting in five minutes.’’ The numerical condition displayed the risk magnitudes in numerical (frequency) form; the graphical condition displayed the number of people affected graphically using stick figures and the number at risk (the background) numerically. In addition, half of the participants were given the risk magnitudes with a common denominator (i.e., all likelihood information was given out of 1,000), whereas half were given the risk magnitudes with a noncommon denominator (e.g., likelihood information for Adenovirus was given out of 500, for Brucellosis out of 5,000). Figure 1 depicts the numerical and graphical displays for the Brucellosis, common denominator condition. In addition, participants received the summary sheet, which they kept throughout the study to help them differentiate the diseases when completing the dependent measures. Participants were given 5 min to review the information about the five diseases and told not to take notes on the information. The disease information was given in alphabetical order (i.e., for adenovirus, brucellosis, crypto, OPC, parvovirus B19). However, because of an experimental error, in one of the conditions the parvovirus information was given before the OPC information. Because this could have led those participants to be at a disadvantage on some understanding items (given that the understanding questions were asked in alphabetical order), all analyses were done on just the first three diseases. After 5 min elapsed, participants turned the packet of information over, although they were allowed to keep the summary sheet face up. They then completed the measures of understanding, followed by the numeracy questionnaire.
Results and Discussion As shown in Table 1, there was reasonably good understanding demonstrated for all measures except for comparing to
known risks, where only 53.5% of the questions were answered correctly (50% was chance). We suspect the problem was with our choice of comparison items. Although we spent considerable effort identifying comparison events whose magnitude would be common knowledge, it appears we were not successful. Part of the difficulty may be our use of an undergraduate sample—participants with more experience with risky activities, such as those used by Halpern and colleagues (1989), might be better able to answer these questions. Because of considerable negative skew for the rankings measure, we winsorized the left-side of this distribution at zero, thereby treating a zero correlation or below as reflecting a lack of understanding. Consistency in Understanding Measures There were positive correlations among all measures, with the exception of comparing to known risks, which was not significantly correlated with any other measure. With this item removed, a factor analysis revealed that all of the measures loaded on one factor (eigenvalue ¼ 2.01, all other eigenvalues .48, 50.3% of the variance explained, average correlation ¼ .32, Cronbach’s alpha ¼ .66 (see Table 2). The following analyses are thus conducted on all dependent measures except for comparing to known risks. Effect of Display Type To equate the influence of each of the four remaining understanding measures, we z-standardized each measure and then conducted a 2 (display format: numerical vs. graphical) 2 (denominator consistency: common denominator vs. noncommon denominator) 2 (numeracy: low vs. high) by 4 (understanding measure) repeated-measures analysis of variance (ANOVA), where understanding measure was the repeated-measures variable. There were significant main effects of each of the between-subjects variables. Participants presented with numerical information understood the
Table 1. Average levels of the understanding measures Average response Understanding measure Recall memory (fill-in-the-blanks) Recognition memory (multiple choice) Rankings: Correlation with actual rankings2 Comparing to known risks Perceived understanding Gist recognition Gist recall (GRcl) GRcl correlated with actual likelihoods Worry correlated with actual likelihoods 1
Experiment 1
Experiment 2 1
M ¼ 3.77 (on a 0–6 scale) M ¼ 84.5% correct M correlation ¼ 0.74; median ¼ 1.0 M ¼ 53.5% correct M ¼ 4.36 (on a 1–5 scale)
M correlation ¼ 0.70; median ¼ 0.90 M ¼ 4.37 (on a 1–5 scale) M ¼ 61.2% correct M ¼ 3.65 (on a 0–10 scale)3 M correlation ¼ 0.734; median ¼ 0.92 M correlation ¼ 0.505; median ¼ 0.67
Of the responses, 53.5% were exactly right, 18.8% were close to correct, and 27.7% were not close to correct. Means are prewinsorization. After winsorization, the means increased to M ¼ 0.80 (Experiment 1) and M ¼ 0.75 (Experiment 2). 3 Of the responses, 25.5% were exactly right; 22.0% were close to correct; and 52.5% were not close to correct. 4 Mean is prewinsorization. After winsorization, the mean increased to M ¼ 0.75. 5 Mean is prewinsorization. After winsorization, the mean increased to M ¼ 0.56. 2
1235
Understanding of Risk Magnitudes Table 2. Factor loadings of the understanding measures
Understanding measure
Experiment 2 factor loadings2
Experiment 1 factor loading1
Absolute factor
Relative factor
.82 .77 .71 .04 .49
.08 .60 .81 .81 .15 .15
.88 .07 .27 .33 .88 .63
Recall memory (fill-in-the-blanks) Recognition memory (multiple choice) Rankings: Correlation with actual rankings Comparing to known risks Perceived understanding Gist recognition Gist recall (GRcl) GRcl correlated with actual likelihoods Worry correlated with actual likelihoods 1
Downloaded by [New York University] at 22:09 03 March 2016
A factor analysis that did not include comparing to known risks produced almost identical results to those shown here, with all loadings within .01 of those shown. 2 All loadings from principal components extraction, varimax rotation.
information better than did participants presented with graphical information, F(1, 297) ¼ 52.51, p < .001; participants presented with a common denominator understood the information better than did participants presented with a noncommon denominator, F(1, 297) ¼ 51.37, p < .001; and participants with higher numeracy demonstrated better understanding than did participants with lower numeracy, F(1, 297) ¼ 6.62, p ¼ .01. There were no interactions among the between-subjects variables (all ps > .29). Thus, the greater understanding produced by the numerical display held regardless of whether a common or noncommon denominator was used and regardless of the participant’s numeracy level. There was, however, a highly significant interaction between display format and understanding measure, indicating that the superiority of the numerical displays was greater for some understanding measures than for
others, multivariate F(3, 295) ¼ 7.31, p < .001. Univariate ANOVAs conducted separately on each of the understanding measures show that the numerical display produced greater understanding than did the graphical display on all of the measures except for comparing to known risks (see Table 3). However, this display effect is particularly strong for the understanding questions that ask about absolute risk. The only other significant interaction was between denominator consistency and understanding measure, multivariate F(3, 295) ¼ 9.09, p < .001. As seen in Table 4, the effect of denominator consistency was particularly strong on rankings, our measure of understanding of the relative risk. Hence, although using noncommon denominators is generally problematic (see Garcia-Retamero et al., 2010; Okan et al., 2012; Paling, 2003), this seems to cause particular difficulty with understanding relative risk magnitudes.
Table 3. Experiment 1 mean understanding levels for numerical and graphical displays Understanding measure Recall memory (fill-in-the-blanks) Recognition memory (multiple choice) Rankings: Correlation with actual rankings Comparing to known risks Perceived understanding
Numerical display
Graphical display
F
P
4.55 91.0% 0.83 54.0% 4.52
3.01 78.0% 0.76 53.1% 4.21
F(1, 306) ¼ 62.82 F(1, 304) ¼ 26.00 F(1, 304) ¼ 3.62 F(1, 303) ¼ 0.43 F(1, 301) ¼ 14.49
