Journal of Clinical Epidemiology 67 (2014) 993e1000
ORIGINAL ARTICLES
Short message service text messaging was feasible as a tool for data collection in a trial of treatment for irritable bowel syndrome Sally Brabyn*, Joy Adamson, Hugh MacPherson, Helen Tilbrook, David J. Torgerson Department of Health Sciences, Seebohm Rowntree Building, University of York, Heslington, York YO10 5DD, UK Accepted 6 May 2014; Published online 25 June 2014
Abstract Objectives: To examine the feasibility of collecting data relating to irritable bowel syndrome (IBS), symptoms by short message service (SMS) text and explore the data to assess its usefulness. Study Design and Setting: In a randomized parallel group design substudy, 59 consenting participants recruited from primary care to a trial of acupuncture for IBS (ISRCTN 08827905) were randomized to receive a one-question SMS message at either 9:30 AM or at 6:30 PM for 7 days: ‘‘On a scale of 0e9, with 0 being no symptoms and 9 being the worst symptoms you could have, how would you score your IBS symptoms now? Please text back a single number.’’ Results: Of the total messages, 59% (n 5 203) were answered within 15 minutes, 73.4% (n 5 254) within 1 hour, and 97% (n 5 334) within 10 hours. Response rates to evening texts were higher (93.5% vs. 87.6% P 5 0.05) and response times shorter though not significantly (median: 0 vs. 5 hours; P 5 0.12). There was no difference in mean scores, and morning symptoms varied more. Mean scores correlated significantly with IBS trial primary outcome measure, the IBS symptom severity score, and secondary outcome measures. Conclusion: Among IBS trial participants, data collection by SMS is feasible and acceptable, and there is potential for deriving meaningful data from the scores. Ó 2014 Elsevier Inc. All rights reserved. Keywords: Short message service (SMS); Text message; Irritable bowel syndrome; Symptom mapping; Data collection; Missing data
1. Introduction The collection of complete outcome data is central to the success of clinical trials, and failure to collect sufficient data, particularly primary outcome data, threatens trial validity and the generalizability of the results. Any loss to follow-up is undesirable; it reduces the study’s statistical power and can introduce bias, if it is systematically different between groups [1]. Follow-up data for trials are often collected in questionnaires that are sent out by post or sent electronically. The challenge lies in getting them back completed. Short Funding: This article presents independent research commissioned by the UK National Institute for Health Research (NIHR)’s Research for Patient Benefit programme (PB-PG-0407-13241). The views expressed in this publication are those of the authors and not necessarily those of the NHS, the NIHR or the UK Department of Health. We have no financial conflict of interest in the publication of this article. This study was unfunded substudy of the Acupuncture for Irritable Bowel Syndrome (ACIBS) trial which was funded by the UK’s National Institute for Health Research’s Research for Patient Benefit program. The main trial registration number is ISRCTN08827905. * Corresponding author. Tel.: (01904)-321-391; fax: (01904) 321651. E-mail address: [email protected] (S. Brabyn). 0895-4356/$ - see front matter Ó 2014 Elsevier Inc. All rights reserved. http://dx.doi.org/10.1016/j.jclinepi.2014.05.004
message service (SMS) technology might offer a new way to enhance data collection. Sending text messages is an enormously popular activity, and SMS is the most widely used mobile data service on earth, with some 700 billion messages reportedly being sent per month in 2007 [2,3]. Indeed more than one billion short messages were sent over China’s networks on February 9 (Chinese New Year), 2005 alone [4], and in the United Kingdom, there are currently 82.7 million mobile phone subscribers who send an average of 200 text messages per month [5] SMS text messaging is straightforward and requires no specialist equipment beyond an entry-level mobile phone. Text messages on mobile phones have been successfully used as reminder tools for appointments [5e7] and medication use [8,9], adjuncts to smoking cessation and weight loss programs [10e13], and have been shown to be acceptable and reliable alternatives to diaries in the management of chronic conditions such as diabetes and asthma [14,15]. The majority of studies of SMS text messaging have focused on its undoubted usefulness as a tool for reminding in one form or another, but the potential it has as a relatively inexpensive easy-to-use tool for data collection in trials is underexplored.
994
S. Brabyn et al. / Journal of Clinical Epidemiology 67 (2014) 993e1000
1.1. Objectives What is new? Key findings Patients, in an irritable bowel syndrome (IBS) study, respond quickly and appropriately to textbased data collection The tendency of participants to respond quickly permits the collection of real-time data Text message IBS symptom scores correlate significantly with other measures What this adds to what was known? Short message service (SMS) text messaging is a feasible, acceptable, and inexpensive method of collecting outcome data in an IBS study and could be an alternative strategy for dealing with missing data What is the implication and what should change now? Clinical studies in the field of IBS should consider using text messaging as a method of collecting outcome data.
Should SMS be a feasible and acceptable method of capturing useful data in clinical trials, its immediacy will have the capacity to minimize problems of recall bias, and its relative cheapness and automation could reduce the costs associated with the use of paper questionnaires in some circumstances. Additionally, the capacity of this method to capture the time of the response could make it possible to collect data from participants at particular times of day or within a specified amount of time of a trial intervention. Time stamping also makes it potentially a useful method for identifying patterns in conditions with intermittent symptoms should appropriate instruments be identified. Single- and two-item health measurement instruments have been found to be valid for use in some clinic settings [16,17] but to our knowledge not delivered by SMS in a trial setting. Recent studies of treatments for low back pain have used SMS text messaging to collect weekly data on numbers of days off work and numbers of days of discomfort [18e20] and have found high levels of adherence that continued for several months enabling researchers and clinicians to map progress and patterns associated with the condition and its treatment [20]. The aim of this study was to find out whether participants in a randomized trial of acupuncture for irritable bowel syndrome (IBS) were prepared to respond to SMS text messages sent daily for a week asking them to score their IBS symptoms and whether they preferred morning or evening texts and to compare the response rates and scores with those of questionnaires.
This was a small-sample study and exploratory in nature. It was designed to explore the potential of SMS text messaging for use as a data collection tool and the possibility that the scores returned by participants could provide a useful measure of IBS symptoms. The objectives were three fold. 1.1.1. Objective 1: feasibility and acceptability To study the feasibility of collecting data on daily symptom scores by SMS text message and assess the relative acceptability of morning or early evening messages, as indicated by the response rate and time taken to respond. Other studies measuring response rates have considered rates between 40% and 89.1% evidence of acceptability [21e24]. Outcome measures: (1) proportion of all SMS messages sent to which a response was received; (2) overall proportion of participants responding to all seven SMS; (3) overall proportion of participants responding to at least one of the seven SMS; and (4) difference in these proportions by morning and evening delivery. 1.1.2. Objective 2: symptom score distribution To explore the distribution of symptom scores and any differences between the morning and evening groups. Outcome measures: (1) mean and standard deviation (SD) of the SMS scores overall and (2) mean and SD of scores by group. 1.1.3. Objective 3: assessment of agreement To assess agreement between the symptom severity scores (SSS) obtained from the text messages and the outcome measures of the larger trial in which this study was nested. Outcome measures: (1) correlation.
2. Methods 2.1. Setting This substudy, a randomized parallel group design, was nested within a larger randomized controlled trial, Acupuncture for IBS [25]. The main trial was a two-group pragmatic randomized controlled trial of acupuncture for the treatment of IBS versus usual General Practitioner care, whose primary aim was to determine the effectiveness and cost effectiveness of acupuncture in the treatment of IBS in primary care. Participants were recruited through database searches of general practices in York, Harrogate, and Knaresborough in Northern England. After screening for eligibility, 233 patients who met the inclusion criteria were randomized into two groups. Participants in the intervention group were offered up to 10 sessions of acupuncture over a 3-month period alongside their
S. Brabyn et al. / Journal of Clinical Epidemiology 67 (2014) 993e1000
usual GP care. The acupuncture was provided at independent clinics by professional acupuncturists trained in traditional Chinese medicine and registered with the British Acupuncture Council. The control group continued with usual GP care. Participants were followed up at 3, 6, 9, and 12 months, with data collected on symptom severity as measured by the IBS SSS [26]. The SSS measures severity and frequency of abdominal pain, severity of abdominal distention, dissatisfaction with bowel habit, and quality of life on a scale of 0e500 (!75 5 no IBS, 75e175 5 mild case, 175e300 5 moderate, and O300 5 severe). Further data were collected using SF12 that provides an overall evaluation of health, with scores ranging from 0 to 100 and also yields two summary scores assessing physical and mental components [27], and the primary outcome for the costeffectiveness component of the study was the EuroQol, EQ5D, a preference-based measure of health state covering five dimensions: mobility, self-care, usual activity, pain and/or discomfort, and anxiety and/or depression [28]. 2.2. Participants The SMS text messaging study participants were a subgroup of the IBS trial participants and as such had a diagnosis of IBS and scored at least 100 on the IBS SSS. The substudy started at the 9-month follow-up time point of the main trial, approximately 6 months after the acupuncture intervention had ended. Potential participants were excluded from the main study and hence from this one if they did not speak English, if they had a diagnosis of cancer or hemophilia or had had recent gastrointestinal surgery, if they were pregnant, or if they were receiving acupuncture. Although having a mobile phone was not a specified criterion for inclusion, it may be that some participants in the main trial were effectively excluded by not having one. Two substudy participants did not have mobile phones but were keen to participate so borrowed them for the study. It was made clear to the participants in the study information that their mobile numbers would be kept confidential and only used for the study and would not be shared with others. 2.3. Sample size This was a convenience sample determined simply by the availability of participants, so no formal sample size calculation was undertaken. This substudy was approved by the York Research Ethics Committee (09/H1311/77) and began recruitment when there were 71 participants in the Acupuncture for IBS trial still to receive 9-month questionnaires. All of them were sent an invitation to participate and 59 (83.1%) responded. Researcher error led to five participants not being entered onto the database, so these participants have been excluded from the analysis.
995
Participants (n 5 3) whose messages were sent out but not delivered for some reason were included in the analysis, so the sample was comprised of 54 (76.1%) participants. 2.3.1. Ethical considerations All participants signed to give informed consent. Only subject identification codes were used in the SMS messaging application and throughout the study. All responses were anonymous and kept in a secure data file. 2.4. SMS intervention Participants received a text message everyday for a 7day period shortly after receiving the 9-month postal questionnaire, so that the data collected by the two methods could be compared. The text of the message was ‘‘On a scale of 1 to 9 with 1 being no symptoms and 9 the worst symptoms you could have, what are your IBS symptoms now? Please text back with a single number.’’ Participants were randomized by an independent data manager to receive the messages at 9:30 in the morning (AM group) or at 6:30 in the evening (PM group). We did this to test the feasibility of collecting data in ‘‘real time,’’ which would not be possible with the traditional paper-based questionnaire, and to explore anecdotal evidence from participants in the main trial that their IBS symptoms varied considerably according to the time of day. Sending the text messages was done by means of a webbased application capable of sending out large numbers of SMS text messages at predetermined times. Participants’ telephone numbers were entered onto the database with their study ID number only, so that anonymity was maintained throughout. The automated SMS system recorded the response as well as both the time the message was sent and the time the response was received. 2.5. Analysis Analysis was conducted using PASW Statistics 18 (SPSS Inc., Chicago, IL, USA). The mean age and baseline SSS scores of the substudy participants were compared with those of the participants in the main trial. They were tested against those not in the substudy using Student t test for independent samples for the SSS scores and the ManneWhitney U test for age. 2.5.1. Objective 1: feasibility and acceptability To explore the feasibility of collecting data on daily symptom scores by SMS text message and to assess the relative acceptability of morning or early evening messages, we measured the differences in response rates and time to respond between the two groups, AM and PM. The data were summarized showing the response rate of each group and the time taken to respond.
996
S. Brabyn et al. / Journal of Clinical Epidemiology 67 (2014) 993e1000
The response rate was measured by (1) total number of replies received/total messages sent (participants days; undeliverable messages included in denominator). Estimates with 95% confidence intervals were noted for proportions of responses to all messages and the difference in the proportions of messages sent to responses received in both morning and evening groups. We also noted the proportion of participants who responded to every message sent and the time between sending the request message and receiving the response.
scores varied a lot over the course of a week and those whose scores did not, participants were grouped according to the range of their scores over the course of a week; those with a range of up to four points were considered to show little variation and those with a range of five or more were considered to have wider variation. The chi-square test was used to test whether there was any association between the time of day and the range of scores, and the two groups were tested for correlation with the main trial primary outcome measure (SSS).
2.5.2. Objective 2: symptom score distribution Participants’ scores were recorded as the mean of the week’s seven daily responses. To compare the severity of symptoms reported in the morning and in the evening, the mean scores of the two groups (AM and PM) were compared using Student independent samples t test and a box plot of participants’ mean SMS scores, comparing morning with evening. What was not captured by looking only at the means is the difference in the range of individual scores making up the participant means. Some participants returned scores that varied little from day to day, whereas others showed considerable variation. To look more closely at this and to see whether there were differences between those whose
2.5.3. Objective 3: assessment of agreement To try and ascertain what, if anything, the SMS scores measure, they were initially tested for correlation with the primary outcome measure used in the main trial: the IBS SSS and the secondary outcome measures SF12, EQ5D, using Kendal tau b.
3. Results The flow of participants is shown in Fig. 1. Of those approached, 83.1% (n 5 59) consented to take part. Simple randomization undertaken by an independent data manager produced groups of 27 and 32 participants. Five
Fig. 1. Flow diagram of recruitment to the substudy. PPTs, participants.
S. Brabyn et al. / Journal of Clinical Epidemiology 67 (2014) 993e1000
997
Table 1. Baseline characteristics of substudy and main trial participants by study arm
Characteristic Age Mean (SD) Median (range) Sex Male, n (%) Baseline mean SSS scores (SD)
Acupuncture (n [ 116) 44.28 (14.31) 43.81 (21.33e78.29) 21 (18.10) 280.00 (81.34), (n 5 115)
Substudy acupuncture (n [ 34) 43.82 (12.46) 43.47 (21.33e67.41) 7 (20.6) 289.24 (76.86)
Usual care (n [ 117)
Substudy usual care (n [ 20)
42.68 (14.79) 42.00 (19.51e74.47) 24 (20.51) 277.17 (71.50), (n 5 113)
42.23 (13.86) 43.16 (25.65e74.09) 3 (15.05) 290.25 (62.78)
Substudy total (n [ 54)
Total (n [ 233) 43.47 (14.54) 42.69 (19.51e78.29) 45 (19.31) 278.60 (76.47)
43.23 (12.89) 43.38 (21.33e74.09) 10 (18.50) 289.61 (71.36)
Abbreviations: SD, standard deviation; SSS, symptom severity score.
participants were not included in the intervention because of an error, so the study sample was made up of 54 participants. The first group (AM, n 5 23) were sent text messages at 9:30 in the morning and the second group (PM, n 5 31) at 6:30 in the evening. 3.1. Baseline Baseline characteristics of main trial participants and those in the study were examined for equivalence (Table 1). The substudy population were similar in all the baseline characteristics, and any small differences may have been attributable to chance. Student t (SSS) and ManneWhitney U (age) tests showed no significant difference in the mean baseline SSS scores and age between those in the substudy and those not, overall and by main trial allocation (Table 2). 3.2. Objective 1: feasibility and acceptability 3.2.1. Response rate Participants were each sent a message a day for seven consecutive days, a total of 378 messages to which 344 responses were received, a 91% response rate. Forty-nine participants received and replied to seven messages and one participant, having responded on the first day, was willing to keep responding but unable to text having lost her telephone. Messages were sent to, and failed to reach, three participants. Only one participant had all messages successfully delivered but did not reply once. The automated SMS system only records that the message was delivered not that it was opened and read, so it is not possible to tell whether the participant received the messages and decided not to reply or there was some other explanation. The SMS service
codes undelivered messages S (undeliverable) or RE (routing error) and either could mean that the telephone is broken or permanently switched off or the number is no longer in use because of a contract change, upgrade, or other reason (Table 3). Answered messages vs. unanswered messages were compared in the AM and PM groups using the chi-square statistic which showed a significant association between time of day and response rate, chi-square 5 4.025, df 5 1, P 5 0.05. 3.2.2. Time taken to respond The time taken for the participants to respond was measured in quarters of an hour from the time the SMS service recorded the message as having been sent. Of the total number of messages, 59% (n 5 203) were answered in under 15 minutes from the time of sending, 73.4% (n 5 254) within 1 hour, and 97% (n 5 334) within 10 hours. There was no significant difference between groups in the time taken to respond, P 5 0.12 (Table 4). 3.3. Objective 2: distribution of symptom scores The participants’ individual mean scores ranged from 0.29 to 7.5 of a possible 9. The mean for the study population was 3.65 (SD, 2.08). Real-time monitoring allowed us to look at differences in symptom patterns in the morning and evening. The difference between the mean scores of the morning and evening groups, 3.36 (standard error [SE], 0.45) and 3.86 (SE, 0.39), respectively, was not significant: t(48) 5 0.86; P O 0.05 (Fig. 2). Of the participants who responded to text messages in the morning, nine (42.9%) returned scores with a range over the week O4, in contrast with eight (27.5%) of those who received messages in the afternoon (Fig. 3).
Table 2. Substudy vs. nonsubstudy participants’ irritable bowel syndrome symptom severity scores at baseline by main trial treatment allocation Confidence intervals Main trial treatment allocation Acupuncture Usual care
t (df) 1.09 (113) 0.84 (110)
Significance 0.275 0.401
Mean difference (standard error) 18.53 (16.9) 15.52 (18.4)
Lower
Upper
52.03 51.9
14.72 7.25
998
S. Brabyn et al. / Journal of Clinical Epidemiology 67 (2014) 993e1000
Table 3. Response rates Proportions
Overall
Responses received to texts sent Confidence interval (CI) Participants responding to all seven short message service (SMS) CI Participants responding to at least one SMS CI
0.9101 0.877, 0.935 0.9074 0.8009, 0.9598 0.9259 0.8244, 0.9708
AM
group
0.8758 0.8159, 0.9181 0.8696 0.6788, 0.9546 0.913 0.732, 0.9758
PM
group
0.9355 0.8919, 0.9629 0.9355 0.7928, 0.9821 0.9355 0.7928, 0.9821
Difference 0.0597 0.0029, 0.1288 0.0659 0.1002, 0.2624 0.0224 0.1334, 0.2094
A chi-square test showed no significant association between time of day and range: chi-square 5 1.65, df 5 1, P 5 0.198. SMS scores with a narrow range correlate better with the SSS scores than those with a wide range (r 5 0.73 and 0.54, respectively).
further explored within the framework of the patientreported outcomes literature. The findings of the study suggest that further examination of both could be worthwhile.
3.4. Objective 3: assessment of agreement
4.1.1. Objective 1: feasibility and acceptability In this article, we have described the feasibility and acceptability of using SMS messaging to collect data as part of a clinical trial. Of those invited 83.1% agreed to receive requests to contribute data about their symptoms via SMS and produced a 91% response rate. Indeed, two participants in the main trial who did not complete their 9-month follow-up postal questionnaires despite two reminders and follow-up telephone calls sent full SMS data about their IBS symptoms, offering the possibility that data collected this way could be used as a basis for imputation to deal with missing questionnaire data. Nested within the overall study, we randomized texts to be sent in the early evening or morning. Messages sent out in the early evening produced a higher response rate (P 5 0.05) and were answered more promptly although the overall difference in response time was not significant, (P 5 0.12). Some participants from the morning group answered around the middle or at the end of the day everyday suggesting that they were waiting for a work break, so an earlier morning time might have produced quite different results. At both time point, participants in this study responded well to the request for symptom scores and they responded promptly, in line with the findings of Anhoj et al. that most of the time people respond to text messages straight away [15].
The SMS text message scores correlate most strongly with the Acupuncture for IBS trial’s primary outcome measure, the IBS SSS (Table 5). With the exception of the SF12 PCS, the SMS scores are more closely correlated with the trial’s secondary outcome measures than are the SSS. That the correlations are statistically significant suggests that the SMS scores are measuring something relevant to the questions the Acupuncture for IBS trial was seeking to answer.
4. Discussion SMS text messaging is inexpensive, quick, and popular and requires only a basic mobile phone. In addition, the data come with time and date information. This was a small-sample exploratory study. Its size and scope mean that it could do no more than explore whether study participants in this population will answer text messages asking for data and how quickly and assess whether data collected in this way could show something useful that is worth looking at in greater depth in appropriately powered studies. It has highlighted two areas of interest: The use of mobile phone and SMS technology as a tool for collecting data and minimizing data loss. This may provide a cost-effective addition to conventional strategies for collecting data and minimizing the risk of bias from missing data. The use of a single-item instrument to measure and map IBS symptom severity, delivered by SMS. This could be Table 4. Mean time to response and range of response times by morning or evening
Time Morning Evening Overall
Number of Mean time taken Range of messages Number of to respond hours response sent responses (standard deviation) times in hours 161 217 378
141 203 344
1.34 (2.16) 1.17 (3.24) 1.25 (2.85)
0e48 0e22.5 0e48
4.1. Main findings
4.1.2. Objective 2: distribution of symptom scores The substudy has produced limited evidence to support anecdotal evidence from participants in the main trial that IBS symptoms are more variable in the morning compared with the afternoons, although the groups are very small. Additionally, our findings illustrate the potential for using SMS text messaging to map symptom patterns in IBS and other conditions. Other studies have used this technology to track the patterns of back pain and mood [20,23]. Using usual postal questionnaires would make it difficult produce these findings.
S. Brabyn et al. / Journal of Clinical Epidemiology 67 (2014) 993e1000
999
Table 5. Kendal tau b correlations of the irritable bowel syndrome (IBS) symptom severity scores (SSS) and short message service (SMS) text scores at the 9-month follow-up point with each other, EQ5D and the physical functioning and mental health scales of the short-form 12 health survey (SF12) IBS SSS Questionnaires IBS SSS EQ5D SF12 Mental component score SF12 Physical component score a b
Fig. 2. Short message service (SMS) scores.
4.1.3. Usefulness That the SMS scores are significantly correlated with the acupuncture for IBS trial’s primary outcome measure, the IBS SSS offers some evidence that these data collected by SMS could provide a valid measure of outcome in this illness. Moreover, when we correlate the SMS outcomes with other measures of health, such as the SF12 and EQ5D, there is tendency for the SMS data to produce better correlations with those than the SSS symptom-specific measure of outcome. Johansson and Wederkop [19] found that back pain sufferers in their study would be willing to answer three questions at a time by SMS. It would be interesting to determine whether the same holds true for other patient populations. Further research into participants’ tolerance for the method and the validity of SMS data is worth consideration.
Kendal tau b
SMS scores
P-value
Kendal tau b
P-value
1.0 0.231b 0.220b
1.0 0.039 0.021
0.481 0.386a 0.333a
!0.001 0.001 0.001
0.194b
0.043
0.132b
0.191
a
Correlation is significant at the 0.01 level (two tailed). Correlation is significant at the 0.05 level (two tailed).
4.2. Strengths and weaknesses This is a study of a novel use of a widely used and readily available technology that could have a useful role to play in trials and more widely. Participants in this substudy expressed enthusiasm, so recruitment was straightforward and adherence good. We were able to recruit a population representative of the larger population of the main trial. There are a number of weaknesses to this study that deserve discussion. Our sample size was relatively small and consequently replication of our study in a larger sample merits consideration. Our data only apply to this population: the participants had a chronic illness, some for many years, which may make them differently disposed to taking part in research than other populations. Their experience with the main trial may also have influenced their willingness to participate in this study, nevertheless the response rates were still surprisingly high and higher than the main study paper questionnaire response rates at the 9-month time point. The time-of-day response rate results could have been very different had the timing of, in particular, the morning text been differentdperhaps before work. Further exploration of optimum messaging times would be useful. Chief among its strengths is its simplicity. It is not ideal to incorporate a substudy after the main trial has started, but concerns about sample size notwithstanding; this has produced some interesting and possibly useful findings. 4.3. Implications
Fig. 3. Distribution of scores.
Trial data can be readily and inexpensively collected by SMS using entry level phones that could have important implications for trials with remote or otherwise hard-toreach populations. A one-sentence question can produce useful data. IBS symptoms vary considerably from day to day but there appears to be less variation later in the day. The willingness of trial participants to return data this way would permit the tracking of symptom trajectories between follow-up points, helping to identify early gains and
1000
S. Brabyn et al. / Journal of Clinical Epidemiology 67 (2014) 993e1000
perhaps the optimum time point for trial data collection in some circumstances. People seem to be remarkably compliant with text messages and it may be with continued use of mobile technology that in some cases, text messaging could replace paper-based data collection.
Acknowledgments The University of York, in its role as sponsor for the study, had no input into study design; in the collection, analysis, and interpretation of data; in the writing of the report; and in the decision to submit the article for publication. References [1] Akl E, Briel M, You J, Lamontagne F, Gangji A, Cukierman-Yaffe T, et al. LOST to follow-up Information in Trials (LOST-IT): a protocol on the potential impact. Trials 2009;10(1):40. [2] Kuntsche E, Robert B. Short message service (SMS) technology in alcohol researchda feasibility study. Alcohol Alcohol 2009;44(4): 423e8. [3] Rogers A, Corbett T, Bramley D, Riddell T, Wills M, Lin R-B, et al. Do you smoke after text? Results of a randomised trial of smoking cessation using mobile phone text messaging. Tob Control 2005;14: 255e61. [4] Yan X, Gong M, Thong JYL. Two tales of one service: user acceptance of short message service (SMS) in Hong Kong and China. Info 2006;8(1):16e28. [5] Ofcom. Available at http://media.ofcom.org.uk/facts/. Accessed February 24, 2014. [6] Downer SR, Meara JG, Da Costa AC, Sethuraman K. SMS text messaging improves outpatient attendance. Aust Health Rev 2006; 30:389e96. [7] Koshy E, Car J, Majeed A. Effectiveness of mobile phone short message service (SMS) reminders for ophthalmology outpatient appointments: observational study. BMC Ophthalmol 2008;8:9. [8] Dunbar PJ, Madigan D, Grohskopf L, Revere D, Woodward J, Minstrell J, et al. A two-way messaging system to enhance antiretroviral adherence. J Am Med Inform Assoc 2003;10:11e5. [9] Pop-Eleches C, Thirumurthy H, Habyarimana JP, Zivin JG, Goldstein MP, de Walque D, et al. Mobile phone technologies improve adherence to antiretroviral treatment in a resource-limited setting: a randomized controlled trial of text message reminders. AIDS 2011;25:825e34. [10] Lester RT, Ritvo P, Mills EJ, Kariri A, Karanja S, Chung MH, et al. Effects of a mobile phone short message service on antiretroviral treatment adherence in Kenya (WelTel Kenya1): a randomised trial. Lancet 2010;376:1838e45. [11] Patrick K, Raab F, Adams MA. A text message-based intervention for weight loss: randomized controlled trial. [electronic article]. J Med Internet Res 2009;11(1):e1.
[12] Obermayer J, Riley W, Asif O, Jean-Mary J. College smokingcessation using cell phone text messaging. J Am Coll Health 2004; 53:71e8. [13] Hurling R, Catt M, DeBoni M, Fairley BW, Hurst T, Murray P, et al. Using Internet and mobile phone technology to deliver an automated physical activity program: randomized controlled trial. J Med Internet Res 2007;9:e7. [14] Ferrer-Roca O, Cardenas A, Diaz-Cardama A, Pulido P. Mobile phone text messaging in the management of diabetes. J Telemed Telecare 2004;10(5):282e5. [15] Anhoj J, Moldrup C. Feasibility of collecting diary data from asthma patients through mobile phones and SMS (short message service): response rate analysis and focus group evaluation from a pilot study. J Med Internet Res 2004;6(4):e42. [16] De Salvo K, Bloser N, Reynolds K, He J, Muntner P. Mortality Prediction with a single general self-Rated health question. J Gen Intern Med 2006;21:267e75. [17] McKevitt C, Dindas R, Wolfe C. Two simple questions to assess outcome after stroke: a European study. Stroke 2001;32(3):681e6. [18] Kongsted A, Leboeuf-Yde C. The Nordic back pain subpopulation program: course patterns established through weekly follow-ups in patients treated for low back pain. Chiropr Osteopat 2009;17:11. [19] Johansen B, Wedderkopp N. Comparison between data obtained though real-time data capture by SMS and a retrospective telephone interview. Chiropr Osteopat 2010;18:10. [20] Axen I, Bodin L, Bergstr€om G, Halasz L, Lange F, L€ovgren PW, et al. The use of weekly text messaging over 6 months was a feasible method for monitoring the clinical course of low back pain in patients seeking chiropractic care. J Clin Epidemiol 2012;65: 454e61. [21] Bexelius C, Merk H, Sandin S, Ekman A, Nyren O, K€uhlmannBerenzon S, et al. SMS versus telephone interviews for epidemiological data collection: feasibility study estimating influenza vaccination coverage in the Swedish population. Eur J Epidemiol 2009;24(2): 73e81. [22] Reid SC, Kauer SD, Dudgeon P, Sanci LA, Shrier LA, Patton GC. A mobile phone program to track young people’s experiences of mood, stress and coping. Soc Psychiatry Psychiatr Epidemiol 2009;44(6): 501e7. [23] Shapiro J, Bauer S, Hamer R, Kordy H, Ward D, Bulik C. Use of text messaging for monitoring sugar-sweetened beverages, physical activity, and screen time in children: a pilot study. J Nutr Educ Behav 2008;40(6):385e91. [24] Roberts A, Gorman A. Short message service for outpatient data collection. Br J Anaesth 2009;102(3):436. [25] MacPherson H, Bland JM, Bloor K, Cox H, Geddes D, Kang’ombe AR, et al. Acupuncture for irritable bowel syndrome: a protocol for a pragmatic randomised controlled trial. BMC Gastroenterol 2010;10:63. [26] Francis C, Morris J, Whorwell P. The irritable bowel severity scoring system: a simple method of monitoring irritable bowel syndrome and its progress. Aliment Pharmacol Ther 1997;11(2):395e402. [27] The EuroQol Group. EuroQolda new facility for the measurement of health-related quality of life. Health Policy 1990;16(3):199e208. [28] Ware JE, Kosinski M, Keller SD. A 12-item short-form health survey: construction of scales and preliminary tests of reliability and validity. Med Care 1996;34:220e33.
ORIGINAL ARTICLES
Short message service text messaging was feasible as a tool for data collection in a trial of treatment for irritable bowel syndrome Sally Brabyn*, Joy Adamson, Hugh MacPherson, Helen Tilbrook, David J. Torgerson Department of Health Sciences, Seebohm Rowntree Building, University of York, Heslington, York YO10 5DD, UK Accepted 6 May 2014; Published online 25 June 2014
Abstract Objectives: To examine the feasibility of collecting data relating to irritable bowel syndrome (IBS), symptoms by short message service (SMS) text and explore the data to assess its usefulness. Study Design and Setting: In a randomized parallel group design substudy, 59 consenting participants recruited from primary care to a trial of acupuncture for IBS (ISRCTN 08827905) were randomized to receive a one-question SMS message at either 9:30 AM or at 6:30 PM for 7 days: ‘‘On a scale of 0e9, with 0 being no symptoms and 9 being the worst symptoms you could have, how would you score your IBS symptoms now? Please text back a single number.’’ Results: Of the total messages, 59% (n 5 203) were answered within 15 minutes, 73.4% (n 5 254) within 1 hour, and 97% (n 5 334) within 10 hours. Response rates to evening texts were higher (93.5% vs. 87.6% P 5 0.05) and response times shorter though not significantly (median: 0 vs. 5 hours; P 5 0.12). There was no difference in mean scores, and morning symptoms varied more. Mean scores correlated significantly with IBS trial primary outcome measure, the IBS symptom severity score, and secondary outcome measures. Conclusion: Among IBS trial participants, data collection by SMS is feasible and acceptable, and there is potential for deriving meaningful data from the scores. Ó 2014 Elsevier Inc. All rights reserved. Keywords: Short message service (SMS); Text message; Irritable bowel syndrome; Symptom mapping; Data collection; Missing data
1. Introduction The collection of complete outcome data is central to the success of clinical trials, and failure to collect sufficient data, particularly primary outcome data, threatens trial validity and the generalizability of the results. Any loss to follow-up is undesirable; it reduces the study’s statistical power and can introduce bias, if it is systematically different between groups [1]. Follow-up data for trials are often collected in questionnaires that are sent out by post or sent electronically. The challenge lies in getting them back completed. Short Funding: This article presents independent research commissioned by the UK National Institute for Health Research (NIHR)’s Research for Patient Benefit programme (PB-PG-0407-13241). The views expressed in this publication are those of the authors and not necessarily those of the NHS, the NIHR or the UK Department of Health. We have no financial conflict of interest in the publication of this article. This study was unfunded substudy of the Acupuncture for Irritable Bowel Syndrome (ACIBS) trial which was funded by the UK’s National Institute for Health Research’s Research for Patient Benefit program. The main trial registration number is ISRCTN08827905. * Corresponding author. Tel.: (01904)-321-391; fax: (01904) 321651. E-mail address: [email protected] (S. Brabyn). 0895-4356/$ - see front matter Ó 2014 Elsevier Inc. All rights reserved. http://dx.doi.org/10.1016/j.jclinepi.2014.05.004
message service (SMS) technology might offer a new way to enhance data collection. Sending text messages is an enormously popular activity, and SMS is the most widely used mobile data service on earth, with some 700 billion messages reportedly being sent per month in 2007 [2,3]. Indeed more than one billion short messages were sent over China’s networks on February 9 (Chinese New Year), 2005 alone [4], and in the United Kingdom, there are currently 82.7 million mobile phone subscribers who send an average of 200 text messages per month [5] SMS text messaging is straightforward and requires no specialist equipment beyond an entry-level mobile phone. Text messages on mobile phones have been successfully used as reminder tools for appointments [5e7] and medication use [8,9], adjuncts to smoking cessation and weight loss programs [10e13], and have been shown to be acceptable and reliable alternatives to diaries in the management of chronic conditions such as diabetes and asthma [14,15]. The majority of studies of SMS text messaging have focused on its undoubted usefulness as a tool for reminding in one form or another, but the potential it has as a relatively inexpensive easy-to-use tool for data collection in trials is underexplored.
994
S. Brabyn et al. / Journal of Clinical Epidemiology 67 (2014) 993e1000
1.1. Objectives What is new? Key findings Patients, in an irritable bowel syndrome (IBS) study, respond quickly and appropriately to textbased data collection The tendency of participants to respond quickly permits the collection of real-time data Text message IBS symptom scores correlate significantly with other measures What this adds to what was known? Short message service (SMS) text messaging is a feasible, acceptable, and inexpensive method of collecting outcome data in an IBS study and could be an alternative strategy for dealing with missing data What is the implication and what should change now? Clinical studies in the field of IBS should consider using text messaging as a method of collecting outcome data.
Should SMS be a feasible and acceptable method of capturing useful data in clinical trials, its immediacy will have the capacity to minimize problems of recall bias, and its relative cheapness and automation could reduce the costs associated with the use of paper questionnaires in some circumstances. Additionally, the capacity of this method to capture the time of the response could make it possible to collect data from participants at particular times of day or within a specified amount of time of a trial intervention. Time stamping also makes it potentially a useful method for identifying patterns in conditions with intermittent symptoms should appropriate instruments be identified. Single- and two-item health measurement instruments have been found to be valid for use in some clinic settings [16,17] but to our knowledge not delivered by SMS in a trial setting. Recent studies of treatments for low back pain have used SMS text messaging to collect weekly data on numbers of days off work and numbers of days of discomfort [18e20] and have found high levels of adherence that continued for several months enabling researchers and clinicians to map progress and patterns associated with the condition and its treatment [20]. The aim of this study was to find out whether participants in a randomized trial of acupuncture for irritable bowel syndrome (IBS) were prepared to respond to SMS text messages sent daily for a week asking them to score their IBS symptoms and whether they preferred morning or evening texts and to compare the response rates and scores with those of questionnaires.
This was a small-sample study and exploratory in nature. It was designed to explore the potential of SMS text messaging for use as a data collection tool and the possibility that the scores returned by participants could provide a useful measure of IBS symptoms. The objectives were three fold. 1.1.1. Objective 1: feasibility and acceptability To study the feasibility of collecting data on daily symptom scores by SMS text message and assess the relative acceptability of morning or early evening messages, as indicated by the response rate and time taken to respond. Other studies measuring response rates have considered rates between 40% and 89.1% evidence of acceptability [21e24]. Outcome measures: (1) proportion of all SMS messages sent to which a response was received; (2) overall proportion of participants responding to all seven SMS; (3) overall proportion of participants responding to at least one of the seven SMS; and (4) difference in these proportions by morning and evening delivery. 1.1.2. Objective 2: symptom score distribution To explore the distribution of symptom scores and any differences between the morning and evening groups. Outcome measures: (1) mean and standard deviation (SD) of the SMS scores overall and (2) mean and SD of scores by group. 1.1.3. Objective 3: assessment of agreement To assess agreement between the symptom severity scores (SSS) obtained from the text messages and the outcome measures of the larger trial in which this study was nested. Outcome measures: (1) correlation.
2. Methods 2.1. Setting This substudy, a randomized parallel group design, was nested within a larger randomized controlled trial, Acupuncture for IBS [25]. The main trial was a two-group pragmatic randomized controlled trial of acupuncture for the treatment of IBS versus usual General Practitioner care, whose primary aim was to determine the effectiveness and cost effectiveness of acupuncture in the treatment of IBS in primary care. Participants were recruited through database searches of general practices in York, Harrogate, and Knaresborough in Northern England. After screening for eligibility, 233 patients who met the inclusion criteria were randomized into two groups. Participants in the intervention group were offered up to 10 sessions of acupuncture over a 3-month period alongside their
S. Brabyn et al. / Journal of Clinical Epidemiology 67 (2014) 993e1000
usual GP care. The acupuncture was provided at independent clinics by professional acupuncturists trained in traditional Chinese medicine and registered with the British Acupuncture Council. The control group continued with usual GP care. Participants were followed up at 3, 6, 9, and 12 months, with data collected on symptom severity as measured by the IBS SSS [26]. The SSS measures severity and frequency of abdominal pain, severity of abdominal distention, dissatisfaction with bowel habit, and quality of life on a scale of 0e500 (!75 5 no IBS, 75e175 5 mild case, 175e300 5 moderate, and O300 5 severe). Further data were collected using SF12 that provides an overall evaluation of health, with scores ranging from 0 to 100 and also yields two summary scores assessing physical and mental components [27], and the primary outcome for the costeffectiveness component of the study was the EuroQol, EQ5D, a preference-based measure of health state covering five dimensions: mobility, self-care, usual activity, pain and/or discomfort, and anxiety and/or depression [28]. 2.2. Participants The SMS text messaging study participants were a subgroup of the IBS trial participants and as such had a diagnosis of IBS and scored at least 100 on the IBS SSS. The substudy started at the 9-month follow-up time point of the main trial, approximately 6 months after the acupuncture intervention had ended. Potential participants were excluded from the main study and hence from this one if they did not speak English, if they had a diagnosis of cancer or hemophilia or had had recent gastrointestinal surgery, if they were pregnant, or if they were receiving acupuncture. Although having a mobile phone was not a specified criterion for inclusion, it may be that some participants in the main trial were effectively excluded by not having one. Two substudy participants did not have mobile phones but were keen to participate so borrowed them for the study. It was made clear to the participants in the study information that their mobile numbers would be kept confidential and only used for the study and would not be shared with others. 2.3. Sample size This was a convenience sample determined simply by the availability of participants, so no formal sample size calculation was undertaken. This substudy was approved by the York Research Ethics Committee (09/H1311/77) and began recruitment when there were 71 participants in the Acupuncture for IBS trial still to receive 9-month questionnaires. All of them were sent an invitation to participate and 59 (83.1%) responded. Researcher error led to five participants not being entered onto the database, so these participants have been excluded from the analysis.
995
Participants (n 5 3) whose messages were sent out but not delivered for some reason were included in the analysis, so the sample was comprised of 54 (76.1%) participants. 2.3.1. Ethical considerations All participants signed to give informed consent. Only subject identification codes were used in the SMS messaging application and throughout the study. All responses were anonymous and kept in a secure data file. 2.4. SMS intervention Participants received a text message everyday for a 7day period shortly after receiving the 9-month postal questionnaire, so that the data collected by the two methods could be compared. The text of the message was ‘‘On a scale of 1 to 9 with 1 being no symptoms and 9 the worst symptoms you could have, what are your IBS symptoms now? Please text back with a single number.’’ Participants were randomized by an independent data manager to receive the messages at 9:30 in the morning (AM group) or at 6:30 in the evening (PM group). We did this to test the feasibility of collecting data in ‘‘real time,’’ which would not be possible with the traditional paper-based questionnaire, and to explore anecdotal evidence from participants in the main trial that their IBS symptoms varied considerably according to the time of day. Sending the text messages was done by means of a webbased application capable of sending out large numbers of SMS text messages at predetermined times. Participants’ telephone numbers were entered onto the database with their study ID number only, so that anonymity was maintained throughout. The automated SMS system recorded the response as well as both the time the message was sent and the time the response was received. 2.5. Analysis Analysis was conducted using PASW Statistics 18 (SPSS Inc., Chicago, IL, USA). The mean age and baseline SSS scores of the substudy participants were compared with those of the participants in the main trial. They were tested against those not in the substudy using Student t test for independent samples for the SSS scores and the ManneWhitney U test for age. 2.5.1. Objective 1: feasibility and acceptability To explore the feasibility of collecting data on daily symptom scores by SMS text message and to assess the relative acceptability of morning or early evening messages, we measured the differences in response rates and time to respond between the two groups, AM and PM. The data were summarized showing the response rate of each group and the time taken to respond.
996
S. Brabyn et al. / Journal of Clinical Epidemiology 67 (2014) 993e1000
The response rate was measured by (1) total number of replies received/total messages sent (participants days; undeliverable messages included in denominator). Estimates with 95% confidence intervals were noted for proportions of responses to all messages and the difference in the proportions of messages sent to responses received in both morning and evening groups. We also noted the proportion of participants who responded to every message sent and the time between sending the request message and receiving the response.
scores varied a lot over the course of a week and those whose scores did not, participants were grouped according to the range of their scores over the course of a week; those with a range of up to four points were considered to show little variation and those with a range of five or more were considered to have wider variation. The chi-square test was used to test whether there was any association between the time of day and the range of scores, and the two groups were tested for correlation with the main trial primary outcome measure (SSS).
2.5.2. Objective 2: symptom score distribution Participants’ scores were recorded as the mean of the week’s seven daily responses. To compare the severity of symptoms reported in the morning and in the evening, the mean scores of the two groups (AM and PM) were compared using Student independent samples t test and a box plot of participants’ mean SMS scores, comparing morning with evening. What was not captured by looking only at the means is the difference in the range of individual scores making up the participant means. Some participants returned scores that varied little from day to day, whereas others showed considerable variation. To look more closely at this and to see whether there were differences between those whose
2.5.3. Objective 3: assessment of agreement To try and ascertain what, if anything, the SMS scores measure, they were initially tested for correlation with the primary outcome measure used in the main trial: the IBS SSS and the secondary outcome measures SF12, EQ5D, using Kendal tau b.
3. Results The flow of participants is shown in Fig. 1. Of those approached, 83.1% (n 5 59) consented to take part. Simple randomization undertaken by an independent data manager produced groups of 27 and 32 participants. Five
Fig. 1. Flow diagram of recruitment to the substudy. PPTs, participants.
S. Brabyn et al. / Journal of Clinical Epidemiology 67 (2014) 993e1000
997
Table 1. Baseline characteristics of substudy and main trial participants by study arm
Characteristic Age Mean (SD) Median (range) Sex Male, n (%) Baseline mean SSS scores (SD)
Acupuncture (n [ 116) 44.28 (14.31) 43.81 (21.33e78.29) 21 (18.10) 280.00 (81.34), (n 5 115)
Substudy acupuncture (n [ 34) 43.82 (12.46) 43.47 (21.33e67.41) 7 (20.6) 289.24 (76.86)
Usual care (n [ 117)
Substudy usual care (n [ 20)
42.68 (14.79) 42.00 (19.51e74.47) 24 (20.51) 277.17 (71.50), (n 5 113)
42.23 (13.86) 43.16 (25.65e74.09) 3 (15.05) 290.25 (62.78)
Substudy total (n [ 54)
Total (n [ 233) 43.47 (14.54) 42.69 (19.51e78.29) 45 (19.31) 278.60 (76.47)
43.23 (12.89) 43.38 (21.33e74.09) 10 (18.50) 289.61 (71.36)
Abbreviations: SD, standard deviation; SSS, symptom severity score.
participants were not included in the intervention because of an error, so the study sample was made up of 54 participants. The first group (AM, n 5 23) were sent text messages at 9:30 in the morning and the second group (PM, n 5 31) at 6:30 in the evening. 3.1. Baseline Baseline characteristics of main trial participants and those in the study were examined for equivalence (Table 1). The substudy population were similar in all the baseline characteristics, and any small differences may have been attributable to chance. Student t (SSS) and ManneWhitney U (age) tests showed no significant difference in the mean baseline SSS scores and age between those in the substudy and those not, overall and by main trial allocation (Table 2). 3.2. Objective 1: feasibility and acceptability 3.2.1. Response rate Participants were each sent a message a day for seven consecutive days, a total of 378 messages to which 344 responses were received, a 91% response rate. Forty-nine participants received and replied to seven messages and one participant, having responded on the first day, was willing to keep responding but unable to text having lost her telephone. Messages were sent to, and failed to reach, three participants. Only one participant had all messages successfully delivered but did not reply once. The automated SMS system only records that the message was delivered not that it was opened and read, so it is not possible to tell whether the participant received the messages and decided not to reply or there was some other explanation. The SMS service
codes undelivered messages S (undeliverable) or RE (routing error) and either could mean that the telephone is broken or permanently switched off or the number is no longer in use because of a contract change, upgrade, or other reason (Table 3). Answered messages vs. unanswered messages were compared in the AM and PM groups using the chi-square statistic which showed a significant association between time of day and response rate, chi-square 5 4.025, df 5 1, P 5 0.05. 3.2.2. Time taken to respond The time taken for the participants to respond was measured in quarters of an hour from the time the SMS service recorded the message as having been sent. Of the total number of messages, 59% (n 5 203) were answered in under 15 minutes from the time of sending, 73.4% (n 5 254) within 1 hour, and 97% (n 5 334) within 10 hours. There was no significant difference between groups in the time taken to respond, P 5 0.12 (Table 4). 3.3. Objective 2: distribution of symptom scores The participants’ individual mean scores ranged from 0.29 to 7.5 of a possible 9. The mean for the study population was 3.65 (SD, 2.08). Real-time monitoring allowed us to look at differences in symptom patterns in the morning and evening. The difference between the mean scores of the morning and evening groups, 3.36 (standard error [SE], 0.45) and 3.86 (SE, 0.39), respectively, was not significant: t(48) 5 0.86; P O 0.05 (Fig. 2). Of the participants who responded to text messages in the morning, nine (42.9%) returned scores with a range over the week O4, in contrast with eight (27.5%) of those who received messages in the afternoon (Fig. 3).
Table 2. Substudy vs. nonsubstudy participants’ irritable bowel syndrome symptom severity scores at baseline by main trial treatment allocation Confidence intervals Main trial treatment allocation Acupuncture Usual care
t (df) 1.09 (113) 0.84 (110)
Significance 0.275 0.401
Mean difference (standard error) 18.53 (16.9) 15.52 (18.4)
Lower
Upper
52.03 51.9
14.72 7.25
998
S. Brabyn et al. / Journal of Clinical Epidemiology 67 (2014) 993e1000
Table 3. Response rates Proportions
Overall
Responses received to texts sent Confidence interval (CI) Participants responding to all seven short message service (SMS) CI Participants responding to at least one SMS CI
0.9101 0.877, 0.935 0.9074 0.8009, 0.9598 0.9259 0.8244, 0.9708
AM
group
0.8758 0.8159, 0.9181 0.8696 0.6788, 0.9546 0.913 0.732, 0.9758
PM
group
0.9355 0.8919, 0.9629 0.9355 0.7928, 0.9821 0.9355 0.7928, 0.9821
Difference 0.0597 0.0029, 0.1288 0.0659 0.1002, 0.2624 0.0224 0.1334, 0.2094
A chi-square test showed no significant association between time of day and range: chi-square 5 1.65, df 5 1, P 5 0.198. SMS scores with a narrow range correlate better with the SSS scores than those with a wide range (r 5 0.73 and 0.54, respectively).
further explored within the framework of the patientreported outcomes literature. The findings of the study suggest that further examination of both could be worthwhile.
3.4. Objective 3: assessment of agreement
4.1.1. Objective 1: feasibility and acceptability In this article, we have described the feasibility and acceptability of using SMS messaging to collect data as part of a clinical trial. Of those invited 83.1% agreed to receive requests to contribute data about their symptoms via SMS and produced a 91% response rate. Indeed, two participants in the main trial who did not complete their 9-month follow-up postal questionnaires despite two reminders and follow-up telephone calls sent full SMS data about their IBS symptoms, offering the possibility that data collected this way could be used as a basis for imputation to deal with missing questionnaire data. Nested within the overall study, we randomized texts to be sent in the early evening or morning. Messages sent out in the early evening produced a higher response rate (P 5 0.05) and were answered more promptly although the overall difference in response time was not significant, (P 5 0.12). Some participants from the morning group answered around the middle or at the end of the day everyday suggesting that they were waiting for a work break, so an earlier morning time might have produced quite different results. At both time point, participants in this study responded well to the request for symptom scores and they responded promptly, in line with the findings of Anhoj et al. that most of the time people respond to text messages straight away [15].
The SMS text message scores correlate most strongly with the Acupuncture for IBS trial’s primary outcome measure, the IBS SSS (Table 5). With the exception of the SF12 PCS, the SMS scores are more closely correlated with the trial’s secondary outcome measures than are the SSS. That the correlations are statistically significant suggests that the SMS scores are measuring something relevant to the questions the Acupuncture for IBS trial was seeking to answer.
4. Discussion SMS text messaging is inexpensive, quick, and popular and requires only a basic mobile phone. In addition, the data come with time and date information. This was a small-sample exploratory study. Its size and scope mean that it could do no more than explore whether study participants in this population will answer text messages asking for data and how quickly and assess whether data collected in this way could show something useful that is worth looking at in greater depth in appropriately powered studies. It has highlighted two areas of interest: The use of mobile phone and SMS technology as a tool for collecting data and minimizing data loss. This may provide a cost-effective addition to conventional strategies for collecting data and minimizing the risk of bias from missing data. The use of a single-item instrument to measure and map IBS symptom severity, delivered by SMS. This could be Table 4. Mean time to response and range of response times by morning or evening
Time Morning Evening Overall
Number of Mean time taken Range of messages Number of to respond hours response sent responses (standard deviation) times in hours 161 217 378
141 203 344
1.34 (2.16) 1.17 (3.24) 1.25 (2.85)
0e48 0e22.5 0e48
4.1. Main findings
4.1.2. Objective 2: distribution of symptom scores The substudy has produced limited evidence to support anecdotal evidence from participants in the main trial that IBS symptoms are more variable in the morning compared with the afternoons, although the groups are very small. Additionally, our findings illustrate the potential for using SMS text messaging to map symptom patterns in IBS and other conditions. Other studies have used this technology to track the patterns of back pain and mood [20,23]. Using usual postal questionnaires would make it difficult produce these findings.
S. Brabyn et al. / Journal of Clinical Epidemiology 67 (2014) 993e1000
999
Table 5. Kendal tau b correlations of the irritable bowel syndrome (IBS) symptom severity scores (SSS) and short message service (SMS) text scores at the 9-month follow-up point with each other, EQ5D and the physical functioning and mental health scales of the short-form 12 health survey (SF12) IBS SSS Questionnaires IBS SSS EQ5D SF12 Mental component score SF12 Physical component score a b
Fig. 2. Short message service (SMS) scores.
4.1.3. Usefulness That the SMS scores are significantly correlated with the acupuncture for IBS trial’s primary outcome measure, the IBS SSS offers some evidence that these data collected by SMS could provide a valid measure of outcome in this illness. Moreover, when we correlate the SMS outcomes with other measures of health, such as the SF12 and EQ5D, there is tendency for the SMS data to produce better correlations with those than the SSS symptom-specific measure of outcome. Johansson and Wederkop [19] found that back pain sufferers in their study would be willing to answer three questions at a time by SMS. It would be interesting to determine whether the same holds true for other patient populations. Further research into participants’ tolerance for the method and the validity of SMS data is worth consideration.
Kendal tau b
SMS scores
P-value
Kendal tau b
P-value
1.0 0.231b 0.220b
1.0 0.039 0.021
0.481 0.386a 0.333a
!0.001 0.001 0.001
0.194b
0.043
0.132b
0.191
a
Correlation is significant at the 0.01 level (two tailed). Correlation is significant at the 0.05 level (two tailed).
4.2. Strengths and weaknesses This is a study of a novel use of a widely used and readily available technology that could have a useful role to play in trials and more widely. Participants in this substudy expressed enthusiasm, so recruitment was straightforward and adherence good. We were able to recruit a population representative of the larger population of the main trial. There are a number of weaknesses to this study that deserve discussion. Our sample size was relatively small and consequently replication of our study in a larger sample merits consideration. Our data only apply to this population: the participants had a chronic illness, some for many years, which may make them differently disposed to taking part in research than other populations. Their experience with the main trial may also have influenced their willingness to participate in this study, nevertheless the response rates were still surprisingly high and higher than the main study paper questionnaire response rates at the 9-month time point. The time-of-day response rate results could have been very different had the timing of, in particular, the morning text been differentdperhaps before work. Further exploration of optimum messaging times would be useful. Chief among its strengths is its simplicity. It is not ideal to incorporate a substudy after the main trial has started, but concerns about sample size notwithstanding; this has produced some interesting and possibly useful findings. 4.3. Implications
Fig. 3. Distribution of scores.
Trial data can be readily and inexpensively collected by SMS using entry level phones that could have important implications for trials with remote or otherwise hard-toreach populations. A one-sentence question can produce useful data. IBS symptoms vary considerably from day to day but there appears to be less variation later in the day. The willingness of trial participants to return data this way would permit the tracking of symptom trajectories between follow-up points, helping to identify early gains and
1000
S. Brabyn et al. / Journal of Clinical Epidemiology 67 (2014) 993e1000
perhaps the optimum time point for trial data collection in some circumstances. People seem to be remarkably compliant with text messages and it may be with continued use of mobile technology that in some cases, text messaging could replace paper-based data collection.
Acknowledgments The University of York, in its role as sponsor for the study, had no input into study design; in the collection, analysis, and interpretation of data; in the writing of the report; and in the decision to submit the article for publication. References [1] Akl E, Briel M, You J, Lamontagne F, Gangji A, Cukierman-Yaffe T, et al. LOST to follow-up Information in Trials (LOST-IT): a protocol on the potential impact. Trials 2009;10(1):40. [2] Kuntsche E, Robert B. Short message service (SMS) technology in alcohol researchda feasibility study. Alcohol Alcohol 2009;44(4): 423e8. [3] Rogers A, Corbett T, Bramley D, Riddell T, Wills M, Lin R-B, et al. Do you smoke after text? Results of a randomised trial of smoking cessation using mobile phone text messaging. Tob Control 2005;14: 255e61. [4] Yan X, Gong M, Thong JYL. Two tales of one service: user acceptance of short message service (SMS) in Hong Kong and China. Info 2006;8(1):16e28. [5] Ofcom. Available at http://media.ofcom.org.uk/facts/. Accessed February 24, 2014. [6] Downer SR, Meara JG, Da Costa AC, Sethuraman K. SMS text messaging improves outpatient attendance. Aust Health Rev 2006; 30:389e96. [7] Koshy E, Car J, Majeed A. Effectiveness of mobile phone short message service (SMS) reminders for ophthalmology outpatient appointments: observational study. BMC Ophthalmol 2008;8:9. [8] Dunbar PJ, Madigan D, Grohskopf L, Revere D, Woodward J, Minstrell J, et al. A two-way messaging system to enhance antiretroviral adherence. J Am Med Inform Assoc 2003;10:11e5. [9] Pop-Eleches C, Thirumurthy H, Habyarimana JP, Zivin JG, Goldstein MP, de Walque D, et al. Mobile phone technologies improve adherence to antiretroviral treatment in a resource-limited setting: a randomized controlled trial of text message reminders. AIDS 2011;25:825e34. [10] Lester RT, Ritvo P, Mills EJ, Kariri A, Karanja S, Chung MH, et al. Effects of a mobile phone short message service on antiretroviral treatment adherence in Kenya (WelTel Kenya1): a randomised trial. Lancet 2010;376:1838e45. [11] Patrick K, Raab F, Adams MA. A text message-based intervention for weight loss: randomized controlled trial. [electronic article]. J Med Internet Res 2009;11(1):e1.
[12] Obermayer J, Riley W, Asif O, Jean-Mary J. College smokingcessation using cell phone text messaging. J Am Coll Health 2004; 53:71e8. [13] Hurling R, Catt M, DeBoni M, Fairley BW, Hurst T, Murray P, et al. Using Internet and mobile phone technology to deliver an automated physical activity program: randomized controlled trial. J Med Internet Res 2007;9:e7. [14] Ferrer-Roca O, Cardenas A, Diaz-Cardama A, Pulido P. Mobile phone text messaging in the management of diabetes. J Telemed Telecare 2004;10(5):282e5. [15] Anhoj J, Moldrup C. Feasibility of collecting diary data from asthma patients through mobile phones and SMS (short message service): response rate analysis and focus group evaluation from a pilot study. J Med Internet Res 2004;6(4):e42. [16] De Salvo K, Bloser N, Reynolds K, He J, Muntner P. Mortality Prediction with a single general self-Rated health question. J Gen Intern Med 2006;21:267e75. [17] McKevitt C, Dindas R, Wolfe C. Two simple questions to assess outcome after stroke: a European study. Stroke 2001;32(3):681e6. [18] Kongsted A, Leboeuf-Yde C. The Nordic back pain subpopulation program: course patterns established through weekly follow-ups in patients treated for low back pain. Chiropr Osteopat 2009;17:11. [19] Johansen B, Wedderkopp N. Comparison between data obtained though real-time data capture by SMS and a retrospective telephone interview. Chiropr Osteopat 2010;18:10. [20] Axen I, Bodin L, Bergstr€om G, Halasz L, Lange F, L€ovgren PW, et al. The use of weekly text messaging over 6 months was a feasible method for monitoring the clinical course of low back pain in patients seeking chiropractic care. J Clin Epidemiol 2012;65: 454e61. [21] Bexelius C, Merk H, Sandin S, Ekman A, Nyren O, K€uhlmannBerenzon S, et al. SMS versus telephone interviews for epidemiological data collection: feasibility study estimating influenza vaccination coverage in the Swedish population. Eur J Epidemiol 2009;24(2): 73e81. [22] Reid SC, Kauer SD, Dudgeon P, Sanci LA, Shrier LA, Patton GC. A mobile phone program to track young people’s experiences of mood, stress and coping. Soc Psychiatry Psychiatr Epidemiol 2009;44(6): 501e7. [23] Shapiro J, Bauer S, Hamer R, Kordy H, Ward D, Bulik C. Use of text messaging for monitoring sugar-sweetened beverages, physical activity, and screen time in children: a pilot study. J Nutr Educ Behav 2008;40(6):385e91. [24] Roberts A, Gorman A. Short message service for outpatient data collection. Br J Anaesth 2009;102(3):436. [25] MacPherson H, Bland JM, Bloor K, Cox H, Geddes D, Kang’ombe AR, et al. Acupuncture for irritable bowel syndrome: a protocol for a pragmatic randomised controlled trial. BMC Gastroenterol 2010;10:63. [26] Francis C, Morris J, Whorwell P. The irritable bowel severity scoring system: a simple method of monitoring irritable bowel syndrome and its progress. Aliment Pharmacol Ther 1997;11(2):395e402. [27] The EuroQol Group. EuroQolda new facility for the measurement of health-related quality of life. Health Policy 1990;16(3):199e208. [28] Ware JE, Kosinski M, Keller SD. A 12-item short-form health survey: construction of scales and preliminary tests of reliability and validity. Med Care 1996;34:220e33.
