This article was downloaded by: [198.91.36.79] On: 27 February 2015, At: 11:05 Publisher: Taylor & Francis Informa Ltd Registered in England and Wales Registered Number: 1072954 Registered office: Mortimer House, 37-41 Mortimer Street, London W1T 3JH, UK
Journal of Health Communication: International Perspectives Publication details, including instructions for authors and subscription information: http://www.tandfonline.com/loi/uhcm20
Health Literacy and Usability of Clinical Trial Search Engines a
a
a
Dina Utami , Timothy W. Bickmore , Barbara Barry & Michael K. Paasche-Orlow
b
a
College of Computer and Information Science, Northeastern University , Boston , Massachusetts , USA b
Section of General Internal Medicine, Department of Medicine , Boston University School of Medicine , Boston , Massachusetts , USA Published online: 14 Oct 2014.
Click for updates To cite this article: Dina Utami , Timothy W. Bickmore , Barbara Barry & Michael K. PaascheOrlow (2014) Health Literacy and Usability of Clinical Trial Search Engines, Journal of Health Communication: International Perspectives, 19:sup2, 190-204, DOI: 10.1080/10810730.2014.938842 To link to this article: http://dx.doi.org/10.1080/10810730.2014.938842
PLEASE SCROLL DOWN FOR ARTICLE Taylor & Francis makes every effort to ensure the accuracy of all the information (the “Content”) contained in the publications on our platform. However, Taylor & Francis, our agents, and our licensors make no representations or warranties whatsoever as to the accuracy, completeness, or suitability for any purpose of the Content. Any opinions and views expressed in this publication are the opinions and views of the authors, and are not the views of or endorsed by Taylor & Francis. The accuracy of the Content should not be relied upon and should be independently verified with primary sources of information. Taylor and Francis shall not be liable for any losses, actions, claims, proceedings, demands, costs, expenses, damages, and other liabilities whatsoever or howsoever caused arising directly or indirectly in connection with, in relation to or arising out of the use of the Content. This article may be used for research, teaching, and private study purposes. Any substantial or systematic reproduction, redistribution, reselling, loan, sub-licensing, systematic supply, or distribution in any form to anyone is expressly forbidden. Terms &
Downloaded by [198.91.36.79] at 11:05 27 February 2015
Conditions of access and use can be found at http://www.tandfonline.com/page/termsand-conditions
Journal of Health Communication, 19:190–204, 2014 Copyright # Taylor & Francis Group, LLC ISSN: 1081-0730 print=1087-0415 online DOI: 10.1080/10810730.2014.938842
Health Literacy and Usability of Clinical Trial Search Engines DINA UTAMI, TIMOTHY W. BICKMORE, AND BARBARA BARRY College of Computer and Information Science, Northeastern University, Boston, Massachusetts, USA
Downloaded by [198.91.36.79] at 11:05 27 February 2015
MICHAEL K. PAASCHE-ORLOW Section of General Internal Medicine, Department of Medicine, Boston University School of Medicine, Boston, Massachusetts, USA Several web-based search engines have been developed to assist individuals to find clinical trials for which they may be interested in volunteering. However, these search engines may be difficult for individuals with low health and computer literacy to navigate. The authors present findings from a usability evaluation of clinical trial search tools with 41 participants across the health and computer literacy spectrum. The study consisted of 3 parts: (a) a usability study of an existing web-based clinical trial search tool; (b) a usability study of a keyword-based clinical trial search tool; and (c) an exploratory study investigating users’ information needs when deciding among 2 or more candidate clinical trials. From the first 2 studies, the authors found that users with low health literacy have difficulty forming queries using keywords and have significantly more difficulty using a standard web-based clinical trial search tool compared with users with adequate health literacy. From the third study, the authors identified the search factors most important to individuals searching for clinical trials and how these varied by health literacy level.
Ensuring adequate enrollment in clinical trials is essential for advancing our scientific knowledge and the development and evaluation of new treatments (Guyatt & Rennie, 2002). Ensuring broad representation in trials is essential for addressing health disparities for underrepresented populations for whom treatments may otherwise go untested. Congress recognized this problem in the 1993 NIH Revitalization Act, which aimed to address unequal access to clinical trials by encouraging the participation of women and minorities in NIH-sponsored research (National Institutes of Health, 1994; NIH Revitalization Act, 1993). Although the situation is improving, representation of women and minorities in clinical trials continues to lag (Gifford, Cunningham, Heslin, & al, 2002; Hoel et al., 2009; Meinert & Gilpin, 2001; Oddone et al., 2004). In addition, enrollment in cancer clinical trials remains low for all groups, especially for those from disadvantaged populations. One study found that 85% of all cancer patients were unaware that there were clinical trials they could participate in (Harris Interactive, 2001), and other studies have demonstrated numerous barriers individuals face in finding trials Address correspondence to Timothy W. Bickmore, College of Computer and Information Science, Northeastern University, WVH 202, 360 Huntington Avenue, Boston, MA 02115, USA. E-mail: [email protected]
190
Downloaded by [198.91.36.79] at 11:05 27 February 2015
Usability of Clinical Trial Search Engines
191
they are eligible for (Ross et al., 1999). In addition, surveys have found that minorities, women, and the elderly remain underrepresented in cancer clinical trials (Murthy, Krumholz, & Gros, 2004). Web-based clinical trial search engines hold the promise of providing universal access to information about clinical trials, and several are now publicly available on the Internet (Atkinson et al., 2008). However, the design of these systems may be further promoting disparities by catering primarily to populations of well-educated individuals with high levels of health literacy, numeracy, and computer literacy. Given prior studies showing significant correlations between race and health literacy (Paasche-Orlow, Parker, Gazmararian, Nielsen-Bohlman, & Rudd, 2005; Paasche-Orlow & Wolf, 2007; Volandes et al., 2008), the search engines currently available may be serving to further disparities in access to research studies. In this work, we explore the usability of clinical trial search engine interfaces now widely available on the web and whether they are usable by individuals with low health literacy. We report the results of three investigations. In the first, we conducted a usability evaluation of a currently available search engine customized for cancer clinical trials, with participants performing standardized tasks. In the second study, we aimed to characterize how individuals would use a free text search box (such as that found on Google’s main search page) to search for clinical trials, and how their search terms and strategies vary by health literacy. Last, we conducted a formative design study to understand what factors were important to participants in choosing among several different trials for which they were eligible.
Method Subjects were recruited and brought to the Human-Computer Interaction laboratory at Northeastern University to participate in all three studies in one session and were compensated $15 for their time. This study was approved by Northeastern University’s institutional review board. Participants Adults (18 years old or older) who were able to speak and read English were eligible. Participants were recruited via online advertisements and direct mailings and fliers targeting community centers and housing complexes in neighborhoods which had high densities of minority populations in Boston, Massachusetts. Measures Health literacy was assessed using the Rapid Estimate of Adult Literacy in Medicine (Davis et al., 1993). Participants were split into adequate and inadequate health literacy groups, using a Rapid Estimate of Adult Literacy in Medicine score of ninth grade and higher, as other authors have done (Lincoln et al., 2006; Lindau, Basu, & Leitsch, 2006; Mancuso & Rincon, 2006; Sudore et al., 2006). Numeracy was assessed using a three-item instrument developed by Schwartz and colleagues (1997). Single self-report scale items were used to assess computer literacy and expertise in using web-based search engines, as well as frequency of online information seeking, email and search engine usage. In the first two studies, user interface behavior, such as mouse movements, keyboard strokes, and webpage navigation, was recorded using Morae software from TechSmith, which captured both interface actions and video and audio of the participant as they worked with the computer. From the video recording, user
192
D. Utami et al.
interface–related errors were coded. In the free-text search study the number of keywords used, and user behavior and verbalization were coded and analyzed. In the clinical trial search engine study, the recordings were analyzed to code successful task completion, task completion time, and the number of nature of user errors. Single-scale items were also used to assess overall satisfaction with the clinical trial search engine, ease of use, desire to continue using the search engine, ease in finding relevant studies in a list of search results, and ease in finding information in a clinical trial description. Procedure
Downloaded by [198.91.36.79] at 11:05 27 February 2015
After we administered informed consent and demographic questionnaires, participants proceeded through each of the following studies. Part I: Usability of a Clinical Trial Search Interface Several clinical trial search tools are available on the internet that provide users with a graphical user interface that lets them specify parametric search criteria, such as medical condition, age, and gender, using graphical user interface controls such as drop-down lists and radio buttons. The purpose of this part of the study was to evaluate the usability of a representative search tool for users across the health and computer literacy spectrums. For Part I, participants were introduced to a currently available clinical trial search engine on the Internet. For the purpose of our study, we used the clinical trial search engine developed by the National Cancer Institute (http://www.cancer.gov/ clinicaltrials/search). They were then given three standardized tasks of increasing complexity to perform using this search engine. To create the standardized tasks, we first created 10 search tasks of varying difficulty and pre-tested them with several research assistants. Three tasks were then selected and ordered by the amount of time required in the pretesting. Each of the search tasks was presented as a fictional individual for whom participants were asked to help find a clinical trial (Table 1). For each task, participants were asked to find at least one study that matched the requested criteria and were given 5 minutes to complete it. We considered a task to be complete only when a participant had successfully retrieved a clinical trial that satisfied the task criteria. Once they had found a trial, we asked them to locate specific information in the search results. Part II: Use of Keywords in Free Text Searches Participants were first asked whether they had used the Web for information searches before. If a participant indicated that they did not know what a search Table 1. Search tasks used in clinical trial search engine study Task number Task 1 Task 2 Task 3
Description Amy is a 66-year-old appendix carcinoma cancer patient. She would like to participate in a clinical trial that is related to her condition. Location of the trial does not matter Joe is 66-year-old and lives in Scottsdale, Arizona. He is a prostate cancer patient who would like to participate in a clinical trial that is related to his condition and where he lives. Rosa is a 70-year-old stage III breast cancer patient and is interested in participating in clinical trial that is for postmenopausal women and uses chemotherapy. Location of the trial does not matter.
Usability of Clinical Trial Search Engines
193
Downloaded by [198.91.36.79] at 11:05 27 February 2015
engine was or had never used one, the research assistant introduced the Google image search page, and demonstrated how it could be used to find a picture. Next, participants were asked to type a few words or phrases in a text box to search for a clinical trial in which they may be interested. For the purpose of the study, we developed our own free text search tool interface which presented users with a text entry box on a web page, recorded the keywords they entered, and did not return any results. We did not use an actual search engine for this part of the study because we did not want users getting distracted by advertisements, erroneous search results, or other distracting content that was irrelevant to the task. Once the participants entered their search words or phrases, they were asked to explain their rationale. Participants were then asked how they would change their query if the search they initially specified found no results. Last, they were asked how they would change their query if the search they specified returned too many results, in order to narrow it down. Part III: Decision Making The purpose of this study was to understand participants’ decision-making processes when deciding among candidate trials and how these might vary by health literacy. Participants were shown three pairs of clinical trial descriptions and asked to choose the one they were most interested in volunteering for, and to provide a justification for their choice. Trial descriptions were paired purposefully to have one or two salient features of comparison, with all other features being approximately the same. Key differentiating feature pairs were as follows: (a) medicine or alternative medicine, (b) long or short trial duration, (c) high or low procedure invasiveness, and (d) high or lower level of perceived procedure discomfort. For data analysis, the interviews were recorded, transcribed, and then coded using grounded theory methodology to find deliberation themes and information-seeking behavior (Strauss, 1987). We assessed participant’s inclusion of the four differentiating features when asked to reflect on their reasons for choosing one trial over another.
Results Study Population Forty-one subjects participated in the study (Table 2). Participants were 48.8 years old (SD ¼ 16), 75.6% female and 63% African American. Approximately one third (34%) had inadequate health literacy. Only 38.5% of the participants graduated from college. All participants owned a cell phone, and only 12.8% had no access to a computer. Also, 5% of participants reported they have never used a computer and 15% reported they have never used a web search engine. We found that in our sample, health literacy was correlated with age (r ¼ 0.39, p < .05), ethnicity (v2[4] ¼ 14.46, p < .01), and educational level (v2[7] ¼ 19.28, p < .01), which conforms to the findings of other studies (Bennett et al., 1998; Brown, Prisuta, Jacobs, & Campbell, 1996). Participants with inadequate health literacy had significantly lower basic numeracy skills and computer and search engine skills than those with adequate literacy. They also reported lower frequency of e-mail usage, Internet surfing, and online health information searching (Mann-Whitney’s U test, p < .05, Table 2). Part I: Usability of a Clinical Trial Search Interface Participants believed that they found an average of only 0.84 trials out of the 3 possible, with low health literacy participants finding significantly fewer compared with those with adequate health literacy (0.43 vs. 1.17, Mann-Whitney U test, p < .05, Table 3). However, when results were checked for accuracy, participants found only 0.53 correct trials on average, with inadequate health literacy participants
194
Age, M SD Female, n (%) Race, n (%) Black White Other Education Less than high school High school More than high school Numeracy, M SD Experience with computers How do you feel about using computers? How do you rate your expertise with web search engines? How often do you surf the Internet? How often do you use email? How often do you use search engines? How often do you look online for health information?
Measure
Table 2. Participant demographics
Always Always Always Always
Never Never Never Never
I never used one.
I’m an expert. I love playing with them. I’m an expert.
Anchor 7
I never used one. I don’t like them.
Anchor 1
3 (21.4) 7 (50.0) 3 (21.4) 0.21 0.42 2.43 (0.94) 2.5 (0.85)
3 (7.3) 11 (26.8) 26 (63.4) 1.12 1.17 2.78 (0.72) 3.11 (0.83)
2.57 2.61 2.49 2.1
(1.1) (1.22) (1.1) (0.92)
1.71 1.43 1.64 1.64
(0.82) (0.65) (0.74) (0.74)
2.14 (1.03)
12 (85.7) 0 (0.0) 2 (14.3)
24 (58.5) 10 (24.4) 7 (17.1)
2.63 (0.86)
44.69 15.66 12 (85.7)
Low literacy (n ¼ 14)
48.81 16 31 (75.6)
All (N ¼ 41)
Downloaded by [198.91.36.79] at 11:05 27 February 2015
3.02 3.22 2.93 2.33
(0.97) (0.97) (0.1) (0.92)
2.89 (0.64)
0 (0.0) 4 (14.8) 23 (85.2) 1.59 1.15 2.96 (0.52) 3.43 (0.63)
12 (44.4) 10 (37.0) 5 (18.5)
57.81 13.1 19 (70.4)
High literacy (n ¼ 27)
Journal of Health Communication: International Perspectives Publication details, including instructions for authors and subscription information: http://www.tandfonline.com/loi/uhcm20
Health Literacy and Usability of Clinical Trial Search Engines a
a
a
Dina Utami , Timothy W. Bickmore , Barbara Barry & Michael K. Paasche-Orlow
b
a
College of Computer and Information Science, Northeastern University , Boston , Massachusetts , USA b
Section of General Internal Medicine, Department of Medicine , Boston University School of Medicine , Boston , Massachusetts , USA Published online: 14 Oct 2014.
Click for updates To cite this article: Dina Utami , Timothy W. Bickmore , Barbara Barry & Michael K. PaascheOrlow (2014) Health Literacy and Usability of Clinical Trial Search Engines, Journal of Health Communication: International Perspectives, 19:sup2, 190-204, DOI: 10.1080/10810730.2014.938842 To link to this article: http://dx.doi.org/10.1080/10810730.2014.938842
PLEASE SCROLL DOWN FOR ARTICLE Taylor & Francis makes every effort to ensure the accuracy of all the information (the “Content”) contained in the publications on our platform. However, Taylor & Francis, our agents, and our licensors make no representations or warranties whatsoever as to the accuracy, completeness, or suitability for any purpose of the Content. Any opinions and views expressed in this publication are the opinions and views of the authors, and are not the views of or endorsed by Taylor & Francis. The accuracy of the Content should not be relied upon and should be independently verified with primary sources of information. Taylor and Francis shall not be liable for any losses, actions, claims, proceedings, demands, costs, expenses, damages, and other liabilities whatsoever or howsoever caused arising directly or indirectly in connection with, in relation to or arising out of the use of the Content. This article may be used for research, teaching, and private study purposes. Any substantial or systematic reproduction, redistribution, reselling, loan, sub-licensing, systematic supply, or distribution in any form to anyone is expressly forbidden. Terms &
Downloaded by [198.91.36.79] at 11:05 27 February 2015
Conditions of access and use can be found at http://www.tandfonline.com/page/termsand-conditions
Journal of Health Communication, 19:190–204, 2014 Copyright # Taylor & Francis Group, LLC ISSN: 1081-0730 print=1087-0415 online DOI: 10.1080/10810730.2014.938842
Health Literacy and Usability of Clinical Trial Search Engines DINA UTAMI, TIMOTHY W. BICKMORE, AND BARBARA BARRY College of Computer and Information Science, Northeastern University, Boston, Massachusetts, USA
Downloaded by [198.91.36.79] at 11:05 27 February 2015
MICHAEL K. PAASCHE-ORLOW Section of General Internal Medicine, Department of Medicine, Boston University School of Medicine, Boston, Massachusetts, USA Several web-based search engines have been developed to assist individuals to find clinical trials for which they may be interested in volunteering. However, these search engines may be difficult for individuals with low health and computer literacy to navigate. The authors present findings from a usability evaluation of clinical trial search tools with 41 participants across the health and computer literacy spectrum. The study consisted of 3 parts: (a) a usability study of an existing web-based clinical trial search tool; (b) a usability study of a keyword-based clinical trial search tool; and (c) an exploratory study investigating users’ information needs when deciding among 2 or more candidate clinical trials. From the first 2 studies, the authors found that users with low health literacy have difficulty forming queries using keywords and have significantly more difficulty using a standard web-based clinical trial search tool compared with users with adequate health literacy. From the third study, the authors identified the search factors most important to individuals searching for clinical trials and how these varied by health literacy level.
Ensuring adequate enrollment in clinical trials is essential for advancing our scientific knowledge and the development and evaluation of new treatments (Guyatt & Rennie, 2002). Ensuring broad representation in trials is essential for addressing health disparities for underrepresented populations for whom treatments may otherwise go untested. Congress recognized this problem in the 1993 NIH Revitalization Act, which aimed to address unequal access to clinical trials by encouraging the participation of women and minorities in NIH-sponsored research (National Institutes of Health, 1994; NIH Revitalization Act, 1993). Although the situation is improving, representation of women and minorities in clinical trials continues to lag (Gifford, Cunningham, Heslin, & al, 2002; Hoel et al., 2009; Meinert & Gilpin, 2001; Oddone et al., 2004). In addition, enrollment in cancer clinical trials remains low for all groups, especially for those from disadvantaged populations. One study found that 85% of all cancer patients were unaware that there were clinical trials they could participate in (Harris Interactive, 2001), and other studies have demonstrated numerous barriers individuals face in finding trials Address correspondence to Timothy W. Bickmore, College of Computer and Information Science, Northeastern University, WVH 202, 360 Huntington Avenue, Boston, MA 02115, USA. E-mail: [email protected]
190
Downloaded by [198.91.36.79] at 11:05 27 February 2015
Usability of Clinical Trial Search Engines
191
they are eligible for (Ross et al., 1999). In addition, surveys have found that minorities, women, and the elderly remain underrepresented in cancer clinical trials (Murthy, Krumholz, & Gros, 2004). Web-based clinical trial search engines hold the promise of providing universal access to information about clinical trials, and several are now publicly available on the Internet (Atkinson et al., 2008). However, the design of these systems may be further promoting disparities by catering primarily to populations of well-educated individuals with high levels of health literacy, numeracy, and computer literacy. Given prior studies showing significant correlations between race and health literacy (Paasche-Orlow, Parker, Gazmararian, Nielsen-Bohlman, & Rudd, 2005; Paasche-Orlow & Wolf, 2007; Volandes et al., 2008), the search engines currently available may be serving to further disparities in access to research studies. In this work, we explore the usability of clinical trial search engine interfaces now widely available on the web and whether they are usable by individuals with low health literacy. We report the results of three investigations. In the first, we conducted a usability evaluation of a currently available search engine customized for cancer clinical trials, with participants performing standardized tasks. In the second study, we aimed to characterize how individuals would use a free text search box (such as that found on Google’s main search page) to search for clinical trials, and how their search terms and strategies vary by health literacy. Last, we conducted a formative design study to understand what factors were important to participants in choosing among several different trials for which they were eligible.
Method Subjects were recruited and brought to the Human-Computer Interaction laboratory at Northeastern University to participate in all three studies in one session and were compensated $15 for their time. This study was approved by Northeastern University’s institutional review board. Participants Adults (18 years old or older) who were able to speak and read English were eligible. Participants were recruited via online advertisements and direct mailings and fliers targeting community centers and housing complexes in neighborhoods which had high densities of minority populations in Boston, Massachusetts. Measures Health literacy was assessed using the Rapid Estimate of Adult Literacy in Medicine (Davis et al., 1993). Participants were split into adequate and inadequate health literacy groups, using a Rapid Estimate of Adult Literacy in Medicine score of ninth grade and higher, as other authors have done (Lincoln et al., 2006; Lindau, Basu, & Leitsch, 2006; Mancuso & Rincon, 2006; Sudore et al., 2006). Numeracy was assessed using a three-item instrument developed by Schwartz and colleagues (1997). Single self-report scale items were used to assess computer literacy and expertise in using web-based search engines, as well as frequency of online information seeking, email and search engine usage. In the first two studies, user interface behavior, such as mouse movements, keyboard strokes, and webpage navigation, was recorded using Morae software from TechSmith, which captured both interface actions and video and audio of the participant as they worked with the computer. From the video recording, user
192
D. Utami et al.
interface–related errors were coded. In the free-text search study the number of keywords used, and user behavior and verbalization were coded and analyzed. In the clinical trial search engine study, the recordings were analyzed to code successful task completion, task completion time, and the number of nature of user errors. Single-scale items were also used to assess overall satisfaction with the clinical trial search engine, ease of use, desire to continue using the search engine, ease in finding relevant studies in a list of search results, and ease in finding information in a clinical trial description. Procedure
Downloaded by [198.91.36.79] at 11:05 27 February 2015
After we administered informed consent and demographic questionnaires, participants proceeded through each of the following studies. Part I: Usability of a Clinical Trial Search Interface Several clinical trial search tools are available on the internet that provide users with a graphical user interface that lets them specify parametric search criteria, such as medical condition, age, and gender, using graphical user interface controls such as drop-down lists and radio buttons. The purpose of this part of the study was to evaluate the usability of a representative search tool for users across the health and computer literacy spectrums. For Part I, participants were introduced to a currently available clinical trial search engine on the Internet. For the purpose of our study, we used the clinical trial search engine developed by the National Cancer Institute (http://www.cancer.gov/ clinicaltrials/search). They were then given three standardized tasks of increasing complexity to perform using this search engine. To create the standardized tasks, we first created 10 search tasks of varying difficulty and pre-tested them with several research assistants. Three tasks were then selected and ordered by the amount of time required in the pretesting. Each of the search tasks was presented as a fictional individual for whom participants were asked to help find a clinical trial (Table 1). For each task, participants were asked to find at least one study that matched the requested criteria and were given 5 minutes to complete it. We considered a task to be complete only when a participant had successfully retrieved a clinical trial that satisfied the task criteria. Once they had found a trial, we asked them to locate specific information in the search results. Part II: Use of Keywords in Free Text Searches Participants were first asked whether they had used the Web for information searches before. If a participant indicated that they did not know what a search Table 1. Search tasks used in clinical trial search engine study Task number Task 1 Task 2 Task 3
Description Amy is a 66-year-old appendix carcinoma cancer patient. She would like to participate in a clinical trial that is related to her condition. Location of the trial does not matter Joe is 66-year-old and lives in Scottsdale, Arizona. He is a prostate cancer patient who would like to participate in a clinical trial that is related to his condition and where he lives. Rosa is a 70-year-old stage III breast cancer patient and is interested in participating in clinical trial that is for postmenopausal women and uses chemotherapy. Location of the trial does not matter.
Usability of Clinical Trial Search Engines
193
Downloaded by [198.91.36.79] at 11:05 27 February 2015
engine was or had never used one, the research assistant introduced the Google image search page, and demonstrated how it could be used to find a picture. Next, participants were asked to type a few words or phrases in a text box to search for a clinical trial in which they may be interested. For the purpose of the study, we developed our own free text search tool interface which presented users with a text entry box on a web page, recorded the keywords they entered, and did not return any results. We did not use an actual search engine for this part of the study because we did not want users getting distracted by advertisements, erroneous search results, or other distracting content that was irrelevant to the task. Once the participants entered their search words or phrases, they were asked to explain their rationale. Participants were then asked how they would change their query if the search they initially specified found no results. Last, they were asked how they would change their query if the search they specified returned too many results, in order to narrow it down. Part III: Decision Making The purpose of this study was to understand participants’ decision-making processes when deciding among candidate trials and how these might vary by health literacy. Participants were shown three pairs of clinical trial descriptions and asked to choose the one they were most interested in volunteering for, and to provide a justification for their choice. Trial descriptions were paired purposefully to have one or two salient features of comparison, with all other features being approximately the same. Key differentiating feature pairs were as follows: (a) medicine or alternative medicine, (b) long or short trial duration, (c) high or low procedure invasiveness, and (d) high or lower level of perceived procedure discomfort. For data analysis, the interviews were recorded, transcribed, and then coded using grounded theory methodology to find deliberation themes and information-seeking behavior (Strauss, 1987). We assessed participant’s inclusion of the four differentiating features when asked to reflect on their reasons for choosing one trial over another.
Results Study Population Forty-one subjects participated in the study (Table 2). Participants were 48.8 years old (SD ¼ 16), 75.6% female and 63% African American. Approximately one third (34%) had inadequate health literacy. Only 38.5% of the participants graduated from college. All participants owned a cell phone, and only 12.8% had no access to a computer. Also, 5% of participants reported they have never used a computer and 15% reported they have never used a web search engine. We found that in our sample, health literacy was correlated with age (r ¼ 0.39, p < .05), ethnicity (v2[4] ¼ 14.46, p < .01), and educational level (v2[7] ¼ 19.28, p < .01), which conforms to the findings of other studies (Bennett et al., 1998; Brown, Prisuta, Jacobs, & Campbell, 1996). Participants with inadequate health literacy had significantly lower basic numeracy skills and computer and search engine skills than those with adequate literacy. They also reported lower frequency of e-mail usage, Internet surfing, and online health information searching (Mann-Whitney’s U test, p < .05, Table 2). Part I: Usability of a Clinical Trial Search Interface Participants believed that they found an average of only 0.84 trials out of the 3 possible, with low health literacy participants finding significantly fewer compared with those with adequate health literacy (0.43 vs. 1.17, Mann-Whitney U test, p < .05, Table 3). However, when results were checked for accuracy, participants found only 0.53 correct trials on average, with inadequate health literacy participants
194
Age, M SD Female, n (%) Race, n (%) Black White Other Education Less than high school High school More than high school Numeracy, M SD Experience with computers How do you feel about using computers? How do you rate your expertise with web search engines? How often do you surf the Internet? How often do you use email? How often do you use search engines? How often do you look online for health information?
Measure
Table 2. Participant demographics
Always Always Always Always
Never Never Never Never
I never used one.
I’m an expert. I love playing with them. I’m an expert.
Anchor 7
I never used one. I don’t like them.
Anchor 1
3 (21.4) 7 (50.0) 3 (21.4) 0.21 0.42 2.43 (0.94) 2.5 (0.85)
3 (7.3) 11 (26.8) 26 (63.4) 1.12 1.17 2.78 (0.72) 3.11 (0.83)
2.57 2.61 2.49 2.1
(1.1) (1.22) (1.1) (0.92)
1.71 1.43 1.64 1.64
(0.82) (0.65) (0.74) (0.74)
2.14 (1.03)
12 (85.7) 0 (0.0) 2 (14.3)
24 (58.5) 10 (24.4) 7 (17.1)
2.63 (0.86)
44.69 15.66 12 (85.7)
Low literacy (n ¼ 14)
48.81 16 31 (75.6)
All (N ¼ 41)
Downloaded by [198.91.36.79] at 11:05 27 February 2015
3.02 3.22 2.93 2.33
(0.97) (0.97) (0.1) (0.92)
2.89 (0.64)
0 (0.0) 4 (14.8) 23 (85.2) 1.59 1.15 2.96 (0.52) 3.43 (0.63)
12 (44.4) 10 (37.0) 5 (18.5)
57.81 13.1 19 (70.4)
High literacy (n ¼ 27)
