RESEARCH ARTICLE
Technology and Information Sharing in Disaster Relief Benedikte Bjerge1, Nathan Clark2*, Peter Fisker1, Emmanuel Raju2 1 Faculty of Science, University of Copenhagen, Copenhagen, Denmark, 2 Faculty of Law, University of Copenhagen, Copenhagen, Denmark * [email protected]
a11111
OPEN ACCESS Citation: Bjerge B, Clark N, Fisker P, Raju E (2016) Technology and Information Sharing in Disaster Relief. PLoS ONE 11(9): e0161783. doi:10.1371/ journal.pone.0161783 Editor: Kim-Kwang Raymond Choo, University of Texas at San Antonio, UNITED STATES Received: June 4, 2016 Accepted: August 11, 2016
Abstract This paper seeks to examine the extent to which technological advances can enhance inter-organizational information sharing in disaster relief. Our case is the Virtual OSOCC (On-Site Operations Coordination Centre) which is a part of the Global Disaster Alert and Coordination System (GDACS) under the United Nations Office for Coordination of Humanitarian Affairs (UN OCHA). The online platform, which has been developing for more than a decade, provides a unique insight into coordination behaviour among disaster management agencies and individual actors. We build our study on the analysis of a complete database of user interaction including more than 20,000 users and 11,000 comments spread across approximately 300 disaster events. Controlling for types and severities of the events, location-specific vulnerabilities, and the overall trends, we find that the introduction of new features have led to increases in user activity. We supplement the data-driven approach with evidence from semi-structured interviews with administrators and key users, as well as a survey among all users specifically designed to capture and assess the elements highlighted by both interviews and data analysis.
Published: September 1, 2016 Copyright: © 2016 Bjerge et al. This is an open access article distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are credited. Data Availability Statement: Access to the data used in this paper was provided to us by Thomas Peter from the Emergency Services Branch at OCHA-Geneva, the unit is responsible for managing the data and storage of system information about users. We have not been given the right to share the data with third parties, but anybody with access to the Virtual OSOCC can access the information about user activities and comments about recent disasters. We will of course be happy to share the code we used for analyzing the data. Funding: The authors received no specific funding for this work.
Introduction The exchange of relevant information is critical in the immediate aftermath of a major disaster. Within the first 72 hours, stakeholders work against the clock to find and rescue survivors, provide life-saving medical treatments, and set up the infrastructure for a long-term humanitarian intervention. However, the disorder of inter-organizational information sharing in this period often leads to overlapping initiatives and the extensive mismanagement of resources, which is in turn linked to the loss of lives and livelihoods on the ground. This is clear from reports following the 2004 Indian Ocean earthquake and tsunami, Hurricane Katrina, and the 2010 Haiti Earthquake ([1];[2];[3]). Fortunately, the advances and access to new technologies have helped progress information sharing efforts in the field. Information communication technologies (ICTs) in particular are changing the way stakeholders communicate and share data within and across borders during crises. Web portals such as ReliefWeb and HumanitarianResponse.info continue to provide up-to-date status reports about ongoing emergencies. At the same time, open source web tools
PLOS ONE | DOI:10.1371/journal.pone.0161783 September 1, 2016
1 / 20
Technology and Information Sharing in Disaster Relief
Competing Interests: The authors have declared that no competing interests exist.
and data deposits are emerging alongside social and technical networks to facilitate more timely and efficient methods for collecting and processing data. In the aftermath of the 2010 Haiti earthquake, members of the affected community and global volunteers used mobile applications and social media networks, such as OpenStreetMaps and Twitter, to coordinate data and information with disaster responders [3]. These collaborations were largely effective, and also worked to bring wider attention to the practices of crowdsourcing, crisis mapping and big data analytics to the traditional relief and response communities [4]. As these technologies continue to advance, we should expect to see an increase in the efficiency of information exchange as well as the amount of information shared among relevant stakeholders in disaster relief networks. This paper aims to assess these effects through an extensive case study of the UN OCHA hosted Virtual On-Site Operations Coordination Centre (Virtual OSOCC). With more than 20,000 registered users within the disaster response community, over the last 15 years, the system has evolved technically and operationally to provide stakeholders with various levels of information in the immediate relief phase following a disaster event. From situational overviews and maps, to the requests and status for relief teams and items, it has become a central information portal for global relief actors. This paper examines the impact of specific changes to the technological architecture and functionality of the Virtual OSOCC on user activity and the amount of information shared over time. We first employ meticulous quantitative methods on a unique data set of complete user interaction and show that the introduction of new features that enhance ease-of-use and usefulness greatly increases the amount of information shared in the system. For instance, putting in place features that help with the organization of information about relief teams and maps have led the number of pieces of information of those types to be tripled and quadrupled respectively. We then support these findings by applying a modified Technology Acceptance Model to data collected through semi-structured interviews and a survey distributed to all active users, which provide useful insights into the perceptions and opinions of users as to the effectiveness of changes made to the system.
Information Sharing in Disaster Relief Coordination Information sharing within the field of disaster relief can be defined within the broader context of coordination. Coordination continues to be a complicated phenomenon, which has different meanings to different stakeholders ([5]; [6]; [7]; [8]; [9]). From the agendas of policymakers, to the operational needs of international and domestic relief entities, the concept is ontologically reshaped by the varying goals and activities of a wide range of actors. For the purposes of this paper, we assent with Malone and Crowstone, who define coordination as “the act of managing interdependencies between activities performed to achieve a goal” [10]. This definition provides that there are various interdependencies between organizations involved in disaster relief and all responding organizations working towards a common goal of saving lives. Within this framework, we can further identify a number of activities which support coordination, such as communication, collaboration, and cooperation [9]. Information sharing falls within the domain of communication, and is also recognized by the IFRC as one of three primary levels of coordination [11] (the other two being collaboration and joint-strategic planning). While all of these activities are in some way interconnected and equally relevant when discussing coordination, it is important to recognize that at the level of information sharing, the exchange of information does not necessarily require or involve any other form of action (see for example; Saab, Maldonado, Orendovici, Tchouakeu, van Gorp, Zhao 2008; Altay and Labonte 2014). In other words, the action of information sharing can be isolated and analyzed separate from its impact on coordination efforts as a whole (ibid). Indeed, that is the approach
PLOS ONE | DOI:10.1371/journal.pone.0161783 September 1, 2016
2 / 20
Technology and Information Sharing in Disaster Relief
taken in this paper. However, to fully understand the implications of information sharing in coordination it is also useful to understand the issues surrounding these interdependencies, and the ways in which technology works to support and/or complicate developments in this area.
Barriers to Information Sharing in the Age of Technology The organized exchange of information in the disaster relief sector has become more complex than ever. Over the past three decades, the swell of aid actors has done little to change the ad hoc ways in which relief organizations coordinate and exchange resources and information on a case-by-case basis [12]. Often, there is more information than necessary and sometimes not the relevant information that a particular stakeholder requires. This process of information sharing and exchange needs to be channelized and coordinated. The challenge therefore . . . is creating an information infrastructure that is sufficiently flexible to manage the dynamic exchange of information among the participating entities in an inter-organizational system, but sufficiently ordered to ensure that the relevant information gets to the responsible parties in valid format and in time to support effective action [13]. One method for achieving this is through the effective use of ICT platforms. ICTs have become a primary asset among stakeholders for the coordination and sharing of information in all phases of the disaster management cycle ([14], [15]). From two-way radios and mobile phones, to humanitarian web forums and social media sites, these platforms function as essential conduits for the timely exchange of information in disaster relief efforts. However, this progression has also resulted in new and constantly changing environments, where diverse groups of actors struggle to function in a coherent manner. The radical increase in stakeholders, along with the emergence of new technical platforms and tools, have created competitive and parallel initiatives, which works against establishing synchronized efforts for information sharing [16]. NGOs, local governments, international institutions, and private entities, each with their own systems and methods for communicating and distributing information, tend to scatter knowledge and create confusion. For instance, even within the disaster management community stakeholders are not always able to distinguish among information channels, such as Reliefweb, Preventionweb, and the UN OCHA Virtual OSOCC. This is problematic because in the first 72 hours following a disaster stakeholders may lose valuable time if they are uncertain where to find—or must look at multiple sources to gather—the relevant information. Additionally, there are major social, legal and ethical issues in information exchange during disasters. This can be seen in patterns of interaction preferred by emergency managers [17] whether they preferred to exchange information using emails and meetings (as a way of having personal contact) or use analytical decision support systems. It is also worth noting the huge digital divide across the world and is certainly important to information exchange in disasters [18]. Further, while in current times there are large amounts of data freely available online [19], it does not become useful unless this data is processed as organized information. In this regard, it is undeniable that not all information goes through a rigorous ethical test during disasters. Other concerns, such as privacy and legal issues around data sharing can also hinder the effective exchange of relevant data in disaster scenarios. Therefore, it is crucial to state that information processing and exchange must adhere to social, ethical and legal norms. While these issues are certainly important to address, in the context of this paper, what is most important is that, there is an absence of understanding into the usability and effectiveness of existing platforms. As these systems evolve, the changes may have important implications for end-users. Their ability to find, process, and share information is directly linked to their understanding and familiarity with the technology. Therefore, this paper asserts that in
PLOS ONE | DOI:10.1371/journal.pone.0161783 September 1, 2016
3 / 20
Technology and Information Sharing in Disaster Relief
Fig 1. Technology Acceptance Model. doi:10.1371/journal.pone.0161783.g001
order to better integrate new technologies and stakeholders into the picture, it is first relevant to assess the effectiveness and limitations of existing platforms from the perspectives of the users.
Technology Acceptance Model On a broader level this paper relates to how and why individuals accept and use technological platforms. The implementation of these instruments (e.g. computer systems, software, email) does not alone ensure the adoption and use of the technology by end-users. Factors such as the design and changes to the technology, as well as the user’s experience, needs, and perceptions, all contribute to the overall use (and performance) of the system. A common method for conceptualizing the causal linkages among these variables is through the application of a Technology Acceptance Model (TAM) ([20]; [21]). The TAM was introduced in the 1980s, and works as a bridge between social psychology with information systems theories for the study of technology adoption by users. In its most basic form, the model suggests that the behavioural intent of users, and the actual system usage, are dependent on both external variables (e.g. system features, time, and experience) and the perceived usefulness and ease-of-use of the technology by the users [20]. For instance, individuals might reject a new web-based system for sharing information if they have difficulty understanding and using the system, or don’t recognize the benefits in doing so. Conversely, if the users perceive the system as easy to use and performance enhancing, their acceptance and use of the system will increase. Since its inception, the TAM has been extended and applied in various dimensions to provide researchers with more predictive methods for understanding and mapping user interactions with technology ([20]; [21]; [22]). For this paper, we have adopted a modified version of the TAM framework to guide our assessment of how changes made to the Virtual OSOCC system (and other external factors) may impact information sharing and coordination among users (see Fig 1). We correlate positive system acceptance and usage to an increase in the amount of shared information, along with the satisfaction of users with the system
PLOS ONE | DOI:10.1371/journal.pone.0161783 September 1, 2016
4 / 20
Technology and Information Sharing in Disaster Relief
performance. While controlling for various factors, we will quantitatively test direct causality between external factors and system use, and validate our results with survey and interview data which reflect how the same variables impact the perceptions and behavioural attitudes of users towards the system. External factors include feature changes in the Virtual OSOCC, time, user experience and professional backgrounds, and alternative systems. We define perceived ease of use by whether users find it easier to access, use, and understand the system based on changes to the aforementioned variables. Finally, we define perceived usefulness as the extent to which users believe that the changes have resulted in more efficient information sharing among registered users.
The Virtual OSOCC A growing need within the international disaster relief community to standardize the coordination of information and efforts among urban search and rescue (USAR) teams after earthquakes prompted the establishment of the International Search and Rescue Advisory Group (INSARAG) in 1991. Within the Emergency Service Branch (ESB) of the United Nations Office of Humanitarian Affairs (UN OCHA), INSARAG developed the concept for an On-Site Operations Coordination Centre (OSOCC) aimed at improving the coordination between local Governments, USAR teams, and other international responders following a disaster. As a ground level mechanism, the OSOCC was an effective initiative. Apart from ReliefWeb, which was launched in 1996 by UN OCHA and acted as a broader, open information sharing tool for all humanitarian networks, a technological platform for addressing specific coordination needs within disaster management sector did not yet exist. As a result, ESB began planning for an affordable, easy to manage solution that could provide real-time information sharing across the globe in the early phases of disasters. The first version of the Virtual OSOCC was developed and launched between 1999 and 2001. Sharing the ReliefWeb server, the system architecture was supported by classic Microsoft applications. The web portal was managed through Active Server Pages (ASP) and ActiveX controls, and data was stored in an Access database on the backend. The technical and operational management of the system was maintained by the system architect and a handful of UN employees and interns, who shuffled these responsibilities in among their other tasks. Because there was no mandate or select budget to operate the system, it was designed to be highly user driven and relied on the voluntary input from stakeholders in the first phases of a disaster. In this context, the initial system primarily functioned as an interactive discussion blog for emergencies, meetings, and training among registered, relevant actors from within the UN and USAR communities. In 2003, a second version of the system was launched through the migration to servers at the UN Offices in Geneva. The migration was necessary as the number of users and need for better processing power within the system began to grow. At the same time, to assist with the management of data and the administration of the web pages, the system architecture was upgraded to a .NET framework with SqlServer 2005. Over the next 12 years the architecture would remain fairly unchanged, with the exception of a migration to SqlServer 2008 and Windows 2012 Webserver in 2015. However, in that timespan there was an increased focus on user behavior and feedback which resulted in various changes to the system features in terms of functionality and usability. The most relevant of these changes included: The integration of the system with the GDACS (Global Disaster Alert and Coordination System) platform, and the implementation of a GLIDE (GLobal IDEntifier) number for all new disaster events, and map feeds (2004); a feature for uploading file attachments with user comments (2005); on-line UNDAC (United Nations Disaster Assessment and Coordination) alerts, a registration feature
PLOS ONE | DOI:10.1371/journal.pone.0161783 September 1, 2016
5 / 20
Technology and Information Sharing in Disaster Relief
for relief teams and relief items; and a separate simulator section to support training and exercises (2006); mandatory acceptance of Terms of Service for all users and the integration of a Hazard Identification Tool (HIT) (2010); a standardized structure for discussions and the introduction of on-line user surveys (2011); a multilingual interface of static content and ongoing improvements to the discussion structure based on feedback from the working groups within the system (2014-Present). Of the changes made to the system over the years, this paper is particularly interested in examining changes in user behavior after the redesign of the systems architecture in 2003, and the integration of the GDACS, maps, attachments, relief teams, and relief items features. The new features were introduced to make it easier for users to interact and share information within the system. For instance, the GDACS alert system sends out text message alerts to all system users when an event occurs. This allows them to closely follow the latest news and situation reports in order to decide within hours how to react. The ability to upload and share maps and other attachments provides all relevant users with centralized access to the same information, such as impact trajectories, damage assessments, and situational overviews. Along similar lines, the introduction of the relief teams and items features give users the opportunity to add, update, and monitor the deployment status of teams and items for each event. Through a standard template, users register general details about the teams and items, and select the appropriate status from a dropdown menu. Status options include monitoring, mobilising, deployed and stand-down for teams, and considered, dispatching, delivered and canceled for items. The current system design is the result of many years of interaction between the administrators and users, where new features are introduced when relevant, and altered or removed if they prove to be unnecessary. In fact, it is this flexible dynamic which has worked to solidify the reputation and usefulness of the system among the disaster management community. Despite being a closed system only accessible to disaster managers, the Virtual OSOCC has attracted more than 20,000 users throughout its lifetime, and this number continues to increase annually by 20 percent. As the leading technical platform for international disaster relief over the last two decades, our analysis of the Virtual OSOCC contributes to a more definitive understanding of the implications of technological advances for stakeholders in the relief sector. The following section introduces the method and results from our analysis of the system.
Methodology The methodology of the study relies on a combination of quantitative and qualitative approaches. We combine a complete activity-log from the Virtual OSOCC database covering the entire lifetime of the system with semi-structured interviews as well as a survey administered to active users. These different data sources enable us to assess how exogenous technological changes to the Virtual OSOCC affect user behavior. To our knowledge, this constitutes the first attempt to systematically study the Virtual OSOCC and to test the importance of technical improvements for inter-organizational information sharing using longitudinal data.
Data collection Our main statistical analysis is based on a complete log of user activity extracted from the Virtual OSOCC server. The data contain information about all users and their corresponding activity in the system for each disaster event created in the Virtual OSOCC since the launch in 2001 up until (including) the Nepal Earthquake on April 25, 2015. In principle all users can create a disaster event; however the vast majority of events created by administrators in response to calls for international assistance.
PLOS ONE | DOI:10.1371/journal.pone.0161783 September 1, 2016
6 / 20
Technology and Information Sharing in Disaster Relief
Fig 2. Number of users and number of events in the Virtual OSOCC. Country colors indicate the number of users; dot size indicates the number of disaster events by country. Country outlines downloaded from naturalearthdata.com. doi:10.1371/journal.pone.0161783.g002
The database includes three layers of information. First, disaster level data includes information about the type of the disaster, the countries hit, the time and date of the creation of the disaster event, the timespan in which the event was active, the number of unique users that actively contributed information regarding the event, the total number of comments made, the number of maps, attachments, relief teams and items, as well as the number of moderators moderating the content of the page. The final sample consists of 215 disaster events. Out of these, earthquakes, floods, and storms constitute the most frequent disaster types recorded with 34, 29 and 26 percent of the events respectively. The second data level includes general information about users including gender, the name of the organization or institution they represent, account creation time, the last time they logged on to the system, and average user statistics such as the total number of comments made and the number of relief teams and items added. We aggregate the user level information by disaster in order to combine it with the disaster-level data. The third layer of data includes all ca. 11,000 comments created by the users throughout the lifetime of the system. For each disaster we observe all actual comments made by each user as well as the time the comment was created. We categorize user comments in order to isolate the effect of software improvements on information sharing specifically. The list of categories is provided in the Supporting Information S1 Table. The comment level data was then merged by category with feature-level data and aggregated to the disaster level allowing us to study the effects on the actual amount of information shared (whether included in a comment or added through a feature) of introducing system features that simplify and sort a certain type of information. Thus the resulting unit of analysis in the quantitative analysis is a disaster. Fig 2 shows the global distribution of events and users. Users of the Virtual OSOCC primarily come from western countries with the United States (1,756), Germany (917), and Australia (650) in the top three. The Asia-Pacific region and Latin America are the regions hardest hit
PLOS ONE | DOI:10.1371/journal.pone.0161783 September 1, 2016
7 / 20
Technology and Information Sharing in Disaster Relief
with 44 and 28 percent of all recorded disasters, respectively. At the country-level, Philippines (18), Indonesia (15), and Haiti (12) have experienced the largest number of recorded disaster events in the Virtual OSOCC. In order to complement the quantitative data and assist with the design and validation of the user survey, we piloted a series of semi-structured interviews with Virtual OSOCC administrators and later a random sample of users. The aim of interviewing first the administrators was to learn from their experiences with the system and establish a baseline for expectations with regards to user behavior. These interviews included a selection of individuals in managerial and operational roles from the GDACS Secretariat and service branches. The administrative interviews took place between February and June 2015 in person and over the phone. User interviews were carried out over the phone or Skype throughout June 2015. Eight individuals were randomly drawn from the entire population of system users. Because we wanted to gauge the experience of the users over the lifetime of the system, we controlled the sample for users that had been registered prior to 2004 and that were currently active (i.e. had logged in and contributed comments to the Virtual OSOCC within the last year). The interview guide reflected on the history of the users and the Virtual OSOCC; user friendliness of the Virtual OSOCC; the purposes and overall performance of the system; different actors using the Virtual OSOCC; experiences particularly related to changes in the system and on the theme of information sharing. These interviews were transcribed and data analyzed by categorizing according to the questions asked. The important issues highlighted in the interviews are presented in the later parts of the paper. As a further supplement to our quantitative findings, we conducted an online survey among the group of users active during the last 5 years. This group consists of around 5,000 users out of whom 1,044 (21 percent) completed the survey. An important aspect of the survey design is that this was done after analyzing the quantitative user data and the semi-structured interviews to ensure consistency and increase relevance between the different data collection methods. For instance, the user survey includes a section on each of the main changes to the Virtual OSOCC with questions regarding the specific change and how this change affected the ease-ofuse and usefulness of the system. These questions were formulated as statements following the Likert scale approach to ordered responses. Descriptive statistics regarding the characteristics of the users surveyed are presented in Table 1. The users are predominantly men: 85 percent of the respondents are male and the Table 1. Descriptive statistics: User survey. Mean
Std. Dev.
0.852
0.356
43.198
9.425
0.672
0.47
Attachments
0.297
0.457
Comments
0.392
0.488
Relief teams
0.441
0.497
Relief items
0.169
0.375
Maps
0.201
0.401
Situation reports
0.332
0.471
Gender (male = 1) Age (years) Registered to receive GDACS alerts What type of information have you contributed to the VO?
Observations: 1,044 doi:10.1371/journal.pone.0161783.t001
PLOS ONE | DOI:10.1371/journal.pone.0161783 September 1, 2016
8 / 20
Technology and Information Sharing in Disaster Relief
median respondent is 43 years old. As expected, the descriptive statistics paint a picture typical of active users. Only 4 percent of the users surveyed state that they never access the Virtual OSOCC, while two-thirds access the system both during and between disaster events. Out of these, some 47 and 45 percent visit the site daily or hourly within the first week of the disaster, respectively. Regarding the type of contributions made to the system by users, the largest share of users have contributed to the system by using the relief team feature, while the second and third most important type of contribution are user comments and situation reports. In the next section we outline our strategy of exploiting the user data taken from the activity-log of the Virtual OSOCC to assess how external factors influence user behavior. We then go on to present first the quantitative and then the qualitative results before concluding.
Estimation strategy In order to test quantitatively the effect of external factors on user behavior, the optimal scenario is to compare actual and counterfactual outcomes (that is, what would have happened to user behavior in the absence of the updates to the system). In other words, one would ideally like to compare how the same users would have behaved with and without the changes to the technology. However, this is not possible since at a given point in time a user cannot have two simultaneous existences. The next best alternative is to compare outcomes of the users exposed to the change to those of a comparison group of unexposed users. Yet, since there exists no comparable technology to the Virtual OSOCC and it is not possible to limit a system intervention to a subset of the users, this approach is not viable. Rather we compare ex post behavior for users with data on their behavior before the system intervention. This approach is also referred to as the reflexive method of impact, where the users’ behavior before the intervention functions as a comparison or control outcome. This method is particularly relevant for the present study as the entire population of users is affected by the system interventions, in which case there is no scope for an external control group. Moreover, since the behavior of the users is observed over the entire life period of the system, structural changes in behavior can be tested for [23]. The main threat to the reflexive method is the presence of other time-varying (external) factors. For instance, imagine that right after the introduction of the new relief teams feature, users of the Virtual OSOCC start announcing a larger share of the actual number of teams deployed in the system. Although this increase in the number of relief teams announced may be due to the system change (through awareness of the usefulness of reporting team deployment), it may also be because the first events following the system change take place in countries considered to be particularly vulnerable or because certain types of disasters tend to generate more activity. To distinguish these effects it is therefore important to account for other factors explaining user behavior. The factors that are likely to affect information sharing can largely be grouped into either disaster specific characteristics or characteristics of the country hit by the disaster. Disaster specific characteristics include the type and severity of the disaster. We account for disaster type as the nature of assistance needed in the immediate aftermath of the disaster depends heavily on the type of the disaster. For instance, earthquakes in urban areas normally require more search and rescue teams compared to a flood. Additionally, more severe disasters are likely to spark more activity in the Virtual OSOCC as the need for assistance generally increases with the severity of the disaster. To account for the severity of disasters we combine the Virtual OSOCC activity log with data from the Emergency Events Database (EM-DAT) on the number of deaths caused by each disaster. A number of recoded disaster events in the Virtual OSOCC are not defined as a disaster in the EM-DAT database. For a disaster to be entered
PLOS ONE | DOI:10.1371/journal.pone.0161783 September 1, 2016
9 / 20
Technology and Information Sharing in Disaster Relief
into the EM-DAT database at least one of the following criteria must be fulfilled: (i) Ten or more people reported killed, (ii) Hundred or more people reported affected, (iii) Declaration of a state of emergency, (iv) call for international assistance. For these cases (20 observations), we complement the information from the database with other data sources, often summarized by Wikipedia. On average, the number of deaths in a disaster event in our sample is 4,406 people. Country specific characteristics include the vulnerability of affected countries and their level of democratization. The vulnerability of the affected country is likely to determine the involvement of the international community and thereby the level and type of activity observed in the Virtual OSOCC. We measure a country’s vulnerability by their Gross National Income (GNI) per capita using the World Bank Indicators. To ensure comparability across countries and years, GNI per capita is converted to a common currency at purchasing power parity. The mean GNI per capita of affected countries is 5,790—well below the global average of about 13,000 in 2010. We further account for the level of democratization since relief actions in some cases has been hindered by the lack of approval by the affected country’s government to enter the country in relation to rescue operations or distribution of water, food and other necessities; for instance during the Cyclone Nargis in Burma in 2008 where the military regime denied workers access to affected areas. The level of democratization is measured using the Polity IV Annual Time-Series database. The data categorize democratic and autocratic patterns of authority and regime changes in all independent countries with total population greater than 500,000 in 2014. For countries smaller than 500,000 inhabitants we use information collected from other sources to categorize the level of democracy. For disasters that hit more than one country we take the population weighted average of the affected countries GNI per capita and democratization index, respectively. The regression that we seek to estimate by OLS in order to evaluate the impact from external factors on user behavior in the Virtual OSOCC can be written as follows: yjt ¼ a þ ychanget þ b1 Djt þ b2 Xit þ gt þ d þ mjit
ð1Þ
where yjt is a vector of outcome measures for disaster j at time t, and the variable changet is an indicator variable for the change made to the Virtual OSOCC. As previously mentioned, the changes we consider in our baseline specification are: i) Re-design (2003), ii) introduction of GDACS alerts (2004), and iii) the introduction of four features: relief teams (2006), relief items (2006), maps (2004), and attachments (2005). The vectors Djt and Xit contain disaster and country specific characteristics, respectively. We also include region specific fixed effects (δ) as well as time fixed effects (γt). We define the regions based on the United Nations’ five geopolitical regional groups: the African Group, the Asian-Pacific Group, the Eastern European Group, the Latin American and Caribbean Group, and finally the Western European and Others Group. Time fixed effects are included in order to account for increasing user activity in the Virtual OSOCC over time. Finally, μjit is the error term containing all unobserved characteristics. We consider nine outcome measures that all approximate different aspects of user activity in the Virtual OSOCC. These outcomes can broadly be categorized into three groups: outcomes related to the overall use of the Virtual OSOCC, outcomes related to user behavior, and outcomes related to specific types of information. The first group of outcomes includes (i) the unique number of active user counts by recorded disaster, and (ii) the number of different organizations/institutions active by disaster. These outcomes will allow us to investigate how the number and type of users are related to the type and severity of the disaster, the vulnerability of the country hit and the type of user activity in the system. The user related outcomes include (iii) the number of comments added during the first day following an event creation,
PLOS ONE | DOI:10.1371/journal.pone.0161783 September 1, 2016
10 / 20
Technology and Information Sharing in Disaster Relief
(iv) the total number of comments created by disaster, and (v) the number of user comments by comment type and disaster. The third group of outcomes relate to specific pieces of information. These are contained in either comments or the different system features introduced in the system. For instance, we investigate whether the number of comments related to relief teams increased after the relief team feature was introduced. Specifically, the third group of outcomes includes the number of user-specific comments about (vi) relief teams, (vii) relief items, (viii) maps, and (viiii) attachments. We finally go even more into detail with subcategories of the relief teams feature.
Quantitative Results Based on a modified version of the TAM framework, we quantitatively test how user behavior related to information sharing changes with technological updates linked to external factors outside the control of the individual user. For this, we start by estimating Eq (1) using the Virtual OSOCC activity log. Results are presented in Table 2. Four dependent variables (yjt) are considered: unique user count, number of users active in disaster event, total number of comments, and the number of comments on the day the disaster event was created. We find that the system re-design in 2003 positively affect the number of users that commented in the system within the first 24 hours after the disaster event was activated in the Virtual OCOSS (column 1). Surprisingly, there is no detectable increase in the number of users that respond within a day compared to before the GDACS alerts system was introduced. This should be seen in relation to our user survey where 67 percent state that they are registered to receive GDACS alerts, and out of these around 75 percent agree that the alerts system helps reduce the time it takes to respond to a disaster event. Besides this, we find that the introduction of GDACS alerts is the system change that has generated the most significant increase in user activity in terms of number of users and comments (columns 1-3). The relief teams and the attachment features both seem to have generated a hike in the number of unique users (column 1). Also, we do not observe an expected decrease in the number of comments in column 3 following the introduction of these features which allow easier information sharing. In other words, although users are now able to share information on for instance relief team deployment by clicking a button instead of writing a comment, they are still active in the debate sections. In contrast, the relief item feature did not increase the number of unique users contributing to a given disaster event (column 1), but did, in line with expectations, decrease the number of comments concerning information related to relief items (column 3). Looking at the control variables, we find that disasters with more fatalities generate more comments and hence, information sharing. The same goes for disasters that happen in countries with a higher per capita GNI, although this relationship varies across information types. The level of democracy—as measured by the Polity IV index—does not seem to have any stand-alone effect on the amount of information shared. Interestingly, user activity is independent of the region in which the event happened (results not reported). Finally, we find that floods and storms/typhoons/cyclones generate less user activity compared to the earthquakes (i.e. the left out category). This is not surprising as earthquakes were initially the event type the system was designed to address. We now change the focus to look at outcomes related to the actual amount of information of different types shared in the system. Since we have categorized all the comments in the database according to the type of information they contain prior to the introduction of specific features, we are able to investigate how different pieces of information types are affected by the
PLOS ONE | DOI:10.1371/journal.pone.0161783 September 1, 2016
11 / 20
Technology and Information Sharing in Disaster Relief
Table 2. Baseline results.
Redesign of VO (= 1) GDACS alerts (= 1) Relief teams (= 1) Relief items (= 1) Attachments (= 1) No. of deaths (log) GNI per capita (log) Democratization Flood Storm/typhoons/cyclones Constant
(1)
(2)
(3)
(4)
Timespan
Unique user count
No. of inst.
No. of comments
1.342***
-0.019
0.184
0.327
(0.449)
(0.114)
(0.252)
(0.277)
0.417
0.400**
0.442*
0.532**
(0.378)
(0.177)
(0.244)
(0.257)
0.520
2.220***
0.160
1.107
(1.021)
(0.470)
(0.947)
(1.312)
-0.119
0.008
-0.783**
-1.492**
(0.543)
(0.278)
(0.394)
(0.655)
-0.007
0.665**
0.445
0.409
(0.653)
(0.306)
(0.557)
(0.697)
0.124***
0.109***
0.220***
0.314***
(0.032)
(0.017)
(0.022)
(0.028)
0.020
0.049
0.068
0.112
(0.087)
(0.034)
(0.066)
(0.076)
-0.007
0.011
0.023*
0.043***
(0.019)
(0.008)
(0.013)
(0.016)
-1.279***
-0.219***
-0.454***
-0.322*
(0.184)
(0.074)
(0.143)
(0.184)
-1.326***
-0.259***
-0.503***
-0.227
(0.179)
(0.084)
(0.143)
(0.174)
0.157
2.646***
0.277
0.608
(0.629)
(0.237)
(0.454)
(0.537)
Region dummies
Yes
Yes
Yes
Yes
Year dummies
Yes
Yes
Yes
Yes
Observations
160
215
215
215
Robust standard errors in parentheses. *** p
Technology and Information Sharing in Disaster Relief Benedikte Bjerge1, Nathan Clark2*, Peter Fisker1, Emmanuel Raju2 1 Faculty of Science, University of Copenhagen, Copenhagen, Denmark, 2 Faculty of Law, University of Copenhagen, Copenhagen, Denmark * [email protected]
a11111
OPEN ACCESS Citation: Bjerge B, Clark N, Fisker P, Raju E (2016) Technology and Information Sharing in Disaster Relief. PLoS ONE 11(9): e0161783. doi:10.1371/ journal.pone.0161783 Editor: Kim-Kwang Raymond Choo, University of Texas at San Antonio, UNITED STATES Received: June 4, 2016 Accepted: August 11, 2016
Abstract This paper seeks to examine the extent to which technological advances can enhance inter-organizational information sharing in disaster relief. Our case is the Virtual OSOCC (On-Site Operations Coordination Centre) which is a part of the Global Disaster Alert and Coordination System (GDACS) under the United Nations Office for Coordination of Humanitarian Affairs (UN OCHA). The online platform, which has been developing for more than a decade, provides a unique insight into coordination behaviour among disaster management agencies and individual actors. We build our study on the analysis of a complete database of user interaction including more than 20,000 users and 11,000 comments spread across approximately 300 disaster events. Controlling for types and severities of the events, location-specific vulnerabilities, and the overall trends, we find that the introduction of new features have led to increases in user activity. We supplement the data-driven approach with evidence from semi-structured interviews with administrators and key users, as well as a survey among all users specifically designed to capture and assess the elements highlighted by both interviews and data analysis.
Published: September 1, 2016 Copyright: © 2016 Bjerge et al. This is an open access article distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are credited. Data Availability Statement: Access to the data used in this paper was provided to us by Thomas Peter from the Emergency Services Branch at OCHA-Geneva, the unit is responsible for managing the data and storage of system information about users. We have not been given the right to share the data with third parties, but anybody with access to the Virtual OSOCC can access the information about user activities and comments about recent disasters. We will of course be happy to share the code we used for analyzing the data. Funding: The authors received no specific funding for this work.
Introduction The exchange of relevant information is critical in the immediate aftermath of a major disaster. Within the first 72 hours, stakeholders work against the clock to find and rescue survivors, provide life-saving medical treatments, and set up the infrastructure for a long-term humanitarian intervention. However, the disorder of inter-organizational information sharing in this period often leads to overlapping initiatives and the extensive mismanagement of resources, which is in turn linked to the loss of lives and livelihoods on the ground. This is clear from reports following the 2004 Indian Ocean earthquake and tsunami, Hurricane Katrina, and the 2010 Haiti Earthquake ([1];[2];[3]). Fortunately, the advances and access to new technologies have helped progress information sharing efforts in the field. Information communication technologies (ICTs) in particular are changing the way stakeholders communicate and share data within and across borders during crises. Web portals such as ReliefWeb and HumanitarianResponse.info continue to provide up-to-date status reports about ongoing emergencies. At the same time, open source web tools
PLOS ONE | DOI:10.1371/journal.pone.0161783 September 1, 2016
1 / 20
Technology and Information Sharing in Disaster Relief
Competing Interests: The authors have declared that no competing interests exist.
and data deposits are emerging alongside social and technical networks to facilitate more timely and efficient methods for collecting and processing data. In the aftermath of the 2010 Haiti earthquake, members of the affected community and global volunteers used mobile applications and social media networks, such as OpenStreetMaps and Twitter, to coordinate data and information with disaster responders [3]. These collaborations were largely effective, and also worked to bring wider attention to the practices of crowdsourcing, crisis mapping and big data analytics to the traditional relief and response communities [4]. As these technologies continue to advance, we should expect to see an increase in the efficiency of information exchange as well as the amount of information shared among relevant stakeholders in disaster relief networks. This paper aims to assess these effects through an extensive case study of the UN OCHA hosted Virtual On-Site Operations Coordination Centre (Virtual OSOCC). With more than 20,000 registered users within the disaster response community, over the last 15 years, the system has evolved technically and operationally to provide stakeholders with various levels of information in the immediate relief phase following a disaster event. From situational overviews and maps, to the requests and status for relief teams and items, it has become a central information portal for global relief actors. This paper examines the impact of specific changes to the technological architecture and functionality of the Virtual OSOCC on user activity and the amount of information shared over time. We first employ meticulous quantitative methods on a unique data set of complete user interaction and show that the introduction of new features that enhance ease-of-use and usefulness greatly increases the amount of information shared in the system. For instance, putting in place features that help with the organization of information about relief teams and maps have led the number of pieces of information of those types to be tripled and quadrupled respectively. We then support these findings by applying a modified Technology Acceptance Model to data collected through semi-structured interviews and a survey distributed to all active users, which provide useful insights into the perceptions and opinions of users as to the effectiveness of changes made to the system.
Information Sharing in Disaster Relief Coordination Information sharing within the field of disaster relief can be defined within the broader context of coordination. Coordination continues to be a complicated phenomenon, which has different meanings to different stakeholders ([5]; [6]; [7]; [8]; [9]). From the agendas of policymakers, to the operational needs of international and domestic relief entities, the concept is ontologically reshaped by the varying goals and activities of a wide range of actors. For the purposes of this paper, we assent with Malone and Crowstone, who define coordination as “the act of managing interdependencies between activities performed to achieve a goal” [10]. This definition provides that there are various interdependencies between organizations involved in disaster relief and all responding organizations working towards a common goal of saving lives. Within this framework, we can further identify a number of activities which support coordination, such as communication, collaboration, and cooperation [9]. Information sharing falls within the domain of communication, and is also recognized by the IFRC as one of three primary levels of coordination [11] (the other two being collaboration and joint-strategic planning). While all of these activities are in some way interconnected and equally relevant when discussing coordination, it is important to recognize that at the level of information sharing, the exchange of information does not necessarily require or involve any other form of action (see for example; Saab, Maldonado, Orendovici, Tchouakeu, van Gorp, Zhao 2008; Altay and Labonte 2014). In other words, the action of information sharing can be isolated and analyzed separate from its impact on coordination efforts as a whole (ibid). Indeed, that is the approach
PLOS ONE | DOI:10.1371/journal.pone.0161783 September 1, 2016
2 / 20
Technology and Information Sharing in Disaster Relief
taken in this paper. However, to fully understand the implications of information sharing in coordination it is also useful to understand the issues surrounding these interdependencies, and the ways in which technology works to support and/or complicate developments in this area.
Barriers to Information Sharing in the Age of Technology The organized exchange of information in the disaster relief sector has become more complex than ever. Over the past three decades, the swell of aid actors has done little to change the ad hoc ways in which relief organizations coordinate and exchange resources and information on a case-by-case basis [12]. Often, there is more information than necessary and sometimes not the relevant information that a particular stakeholder requires. This process of information sharing and exchange needs to be channelized and coordinated. The challenge therefore . . . is creating an information infrastructure that is sufficiently flexible to manage the dynamic exchange of information among the participating entities in an inter-organizational system, but sufficiently ordered to ensure that the relevant information gets to the responsible parties in valid format and in time to support effective action [13]. One method for achieving this is through the effective use of ICT platforms. ICTs have become a primary asset among stakeholders for the coordination and sharing of information in all phases of the disaster management cycle ([14], [15]). From two-way radios and mobile phones, to humanitarian web forums and social media sites, these platforms function as essential conduits for the timely exchange of information in disaster relief efforts. However, this progression has also resulted in new and constantly changing environments, where diverse groups of actors struggle to function in a coherent manner. The radical increase in stakeholders, along with the emergence of new technical platforms and tools, have created competitive and parallel initiatives, which works against establishing synchronized efforts for information sharing [16]. NGOs, local governments, international institutions, and private entities, each with their own systems and methods for communicating and distributing information, tend to scatter knowledge and create confusion. For instance, even within the disaster management community stakeholders are not always able to distinguish among information channels, such as Reliefweb, Preventionweb, and the UN OCHA Virtual OSOCC. This is problematic because in the first 72 hours following a disaster stakeholders may lose valuable time if they are uncertain where to find—or must look at multiple sources to gather—the relevant information. Additionally, there are major social, legal and ethical issues in information exchange during disasters. This can be seen in patterns of interaction preferred by emergency managers [17] whether they preferred to exchange information using emails and meetings (as a way of having personal contact) or use analytical decision support systems. It is also worth noting the huge digital divide across the world and is certainly important to information exchange in disasters [18]. Further, while in current times there are large amounts of data freely available online [19], it does not become useful unless this data is processed as organized information. In this regard, it is undeniable that not all information goes through a rigorous ethical test during disasters. Other concerns, such as privacy and legal issues around data sharing can also hinder the effective exchange of relevant data in disaster scenarios. Therefore, it is crucial to state that information processing and exchange must adhere to social, ethical and legal norms. While these issues are certainly important to address, in the context of this paper, what is most important is that, there is an absence of understanding into the usability and effectiveness of existing platforms. As these systems evolve, the changes may have important implications for end-users. Their ability to find, process, and share information is directly linked to their understanding and familiarity with the technology. Therefore, this paper asserts that in
PLOS ONE | DOI:10.1371/journal.pone.0161783 September 1, 2016
3 / 20
Technology and Information Sharing in Disaster Relief
Fig 1. Technology Acceptance Model. doi:10.1371/journal.pone.0161783.g001
order to better integrate new technologies and stakeholders into the picture, it is first relevant to assess the effectiveness and limitations of existing platforms from the perspectives of the users.
Technology Acceptance Model On a broader level this paper relates to how and why individuals accept and use technological platforms. The implementation of these instruments (e.g. computer systems, software, email) does not alone ensure the adoption and use of the technology by end-users. Factors such as the design and changes to the technology, as well as the user’s experience, needs, and perceptions, all contribute to the overall use (and performance) of the system. A common method for conceptualizing the causal linkages among these variables is through the application of a Technology Acceptance Model (TAM) ([20]; [21]). The TAM was introduced in the 1980s, and works as a bridge between social psychology with information systems theories for the study of technology adoption by users. In its most basic form, the model suggests that the behavioural intent of users, and the actual system usage, are dependent on both external variables (e.g. system features, time, and experience) and the perceived usefulness and ease-of-use of the technology by the users [20]. For instance, individuals might reject a new web-based system for sharing information if they have difficulty understanding and using the system, or don’t recognize the benefits in doing so. Conversely, if the users perceive the system as easy to use and performance enhancing, their acceptance and use of the system will increase. Since its inception, the TAM has been extended and applied in various dimensions to provide researchers with more predictive methods for understanding and mapping user interactions with technology ([20]; [21]; [22]). For this paper, we have adopted a modified version of the TAM framework to guide our assessment of how changes made to the Virtual OSOCC system (and other external factors) may impact information sharing and coordination among users (see Fig 1). We correlate positive system acceptance and usage to an increase in the amount of shared information, along with the satisfaction of users with the system
PLOS ONE | DOI:10.1371/journal.pone.0161783 September 1, 2016
4 / 20
Technology and Information Sharing in Disaster Relief
performance. While controlling for various factors, we will quantitatively test direct causality between external factors and system use, and validate our results with survey and interview data which reflect how the same variables impact the perceptions and behavioural attitudes of users towards the system. External factors include feature changes in the Virtual OSOCC, time, user experience and professional backgrounds, and alternative systems. We define perceived ease of use by whether users find it easier to access, use, and understand the system based on changes to the aforementioned variables. Finally, we define perceived usefulness as the extent to which users believe that the changes have resulted in more efficient information sharing among registered users.
The Virtual OSOCC A growing need within the international disaster relief community to standardize the coordination of information and efforts among urban search and rescue (USAR) teams after earthquakes prompted the establishment of the International Search and Rescue Advisory Group (INSARAG) in 1991. Within the Emergency Service Branch (ESB) of the United Nations Office of Humanitarian Affairs (UN OCHA), INSARAG developed the concept for an On-Site Operations Coordination Centre (OSOCC) aimed at improving the coordination between local Governments, USAR teams, and other international responders following a disaster. As a ground level mechanism, the OSOCC was an effective initiative. Apart from ReliefWeb, which was launched in 1996 by UN OCHA and acted as a broader, open information sharing tool for all humanitarian networks, a technological platform for addressing specific coordination needs within disaster management sector did not yet exist. As a result, ESB began planning for an affordable, easy to manage solution that could provide real-time information sharing across the globe in the early phases of disasters. The first version of the Virtual OSOCC was developed and launched between 1999 and 2001. Sharing the ReliefWeb server, the system architecture was supported by classic Microsoft applications. The web portal was managed through Active Server Pages (ASP) and ActiveX controls, and data was stored in an Access database on the backend. The technical and operational management of the system was maintained by the system architect and a handful of UN employees and interns, who shuffled these responsibilities in among their other tasks. Because there was no mandate or select budget to operate the system, it was designed to be highly user driven and relied on the voluntary input from stakeholders in the first phases of a disaster. In this context, the initial system primarily functioned as an interactive discussion blog for emergencies, meetings, and training among registered, relevant actors from within the UN and USAR communities. In 2003, a second version of the system was launched through the migration to servers at the UN Offices in Geneva. The migration was necessary as the number of users and need for better processing power within the system began to grow. At the same time, to assist with the management of data and the administration of the web pages, the system architecture was upgraded to a .NET framework with SqlServer 2005. Over the next 12 years the architecture would remain fairly unchanged, with the exception of a migration to SqlServer 2008 and Windows 2012 Webserver in 2015. However, in that timespan there was an increased focus on user behavior and feedback which resulted in various changes to the system features in terms of functionality and usability. The most relevant of these changes included: The integration of the system with the GDACS (Global Disaster Alert and Coordination System) platform, and the implementation of a GLIDE (GLobal IDEntifier) number for all new disaster events, and map feeds (2004); a feature for uploading file attachments with user comments (2005); on-line UNDAC (United Nations Disaster Assessment and Coordination) alerts, a registration feature
PLOS ONE | DOI:10.1371/journal.pone.0161783 September 1, 2016
5 / 20
Technology and Information Sharing in Disaster Relief
for relief teams and relief items; and a separate simulator section to support training and exercises (2006); mandatory acceptance of Terms of Service for all users and the integration of a Hazard Identification Tool (HIT) (2010); a standardized structure for discussions and the introduction of on-line user surveys (2011); a multilingual interface of static content and ongoing improvements to the discussion structure based on feedback from the working groups within the system (2014-Present). Of the changes made to the system over the years, this paper is particularly interested in examining changes in user behavior after the redesign of the systems architecture in 2003, and the integration of the GDACS, maps, attachments, relief teams, and relief items features. The new features were introduced to make it easier for users to interact and share information within the system. For instance, the GDACS alert system sends out text message alerts to all system users when an event occurs. This allows them to closely follow the latest news and situation reports in order to decide within hours how to react. The ability to upload and share maps and other attachments provides all relevant users with centralized access to the same information, such as impact trajectories, damage assessments, and situational overviews. Along similar lines, the introduction of the relief teams and items features give users the opportunity to add, update, and monitor the deployment status of teams and items for each event. Through a standard template, users register general details about the teams and items, and select the appropriate status from a dropdown menu. Status options include monitoring, mobilising, deployed and stand-down for teams, and considered, dispatching, delivered and canceled for items. The current system design is the result of many years of interaction between the administrators and users, where new features are introduced when relevant, and altered or removed if they prove to be unnecessary. In fact, it is this flexible dynamic which has worked to solidify the reputation and usefulness of the system among the disaster management community. Despite being a closed system only accessible to disaster managers, the Virtual OSOCC has attracted more than 20,000 users throughout its lifetime, and this number continues to increase annually by 20 percent. As the leading technical platform for international disaster relief over the last two decades, our analysis of the Virtual OSOCC contributes to a more definitive understanding of the implications of technological advances for stakeholders in the relief sector. The following section introduces the method and results from our analysis of the system.
Methodology The methodology of the study relies on a combination of quantitative and qualitative approaches. We combine a complete activity-log from the Virtual OSOCC database covering the entire lifetime of the system with semi-structured interviews as well as a survey administered to active users. These different data sources enable us to assess how exogenous technological changes to the Virtual OSOCC affect user behavior. To our knowledge, this constitutes the first attempt to systematically study the Virtual OSOCC and to test the importance of technical improvements for inter-organizational information sharing using longitudinal data.
Data collection Our main statistical analysis is based on a complete log of user activity extracted from the Virtual OSOCC server. The data contain information about all users and their corresponding activity in the system for each disaster event created in the Virtual OSOCC since the launch in 2001 up until (including) the Nepal Earthquake on April 25, 2015. In principle all users can create a disaster event; however the vast majority of events created by administrators in response to calls for international assistance.
PLOS ONE | DOI:10.1371/journal.pone.0161783 September 1, 2016
6 / 20
Technology and Information Sharing in Disaster Relief
Fig 2. Number of users and number of events in the Virtual OSOCC. Country colors indicate the number of users; dot size indicates the number of disaster events by country. Country outlines downloaded from naturalearthdata.com. doi:10.1371/journal.pone.0161783.g002
The database includes three layers of information. First, disaster level data includes information about the type of the disaster, the countries hit, the time and date of the creation of the disaster event, the timespan in which the event was active, the number of unique users that actively contributed information regarding the event, the total number of comments made, the number of maps, attachments, relief teams and items, as well as the number of moderators moderating the content of the page. The final sample consists of 215 disaster events. Out of these, earthquakes, floods, and storms constitute the most frequent disaster types recorded with 34, 29 and 26 percent of the events respectively. The second data level includes general information about users including gender, the name of the organization or institution they represent, account creation time, the last time they logged on to the system, and average user statistics such as the total number of comments made and the number of relief teams and items added. We aggregate the user level information by disaster in order to combine it with the disaster-level data. The third layer of data includes all ca. 11,000 comments created by the users throughout the lifetime of the system. For each disaster we observe all actual comments made by each user as well as the time the comment was created. We categorize user comments in order to isolate the effect of software improvements on information sharing specifically. The list of categories is provided in the Supporting Information S1 Table. The comment level data was then merged by category with feature-level data and aggregated to the disaster level allowing us to study the effects on the actual amount of information shared (whether included in a comment or added through a feature) of introducing system features that simplify and sort a certain type of information. Thus the resulting unit of analysis in the quantitative analysis is a disaster. Fig 2 shows the global distribution of events and users. Users of the Virtual OSOCC primarily come from western countries with the United States (1,756), Germany (917), and Australia (650) in the top three. The Asia-Pacific region and Latin America are the regions hardest hit
PLOS ONE | DOI:10.1371/journal.pone.0161783 September 1, 2016
7 / 20
Technology and Information Sharing in Disaster Relief
with 44 and 28 percent of all recorded disasters, respectively. At the country-level, Philippines (18), Indonesia (15), and Haiti (12) have experienced the largest number of recorded disaster events in the Virtual OSOCC. In order to complement the quantitative data and assist with the design and validation of the user survey, we piloted a series of semi-structured interviews with Virtual OSOCC administrators and later a random sample of users. The aim of interviewing first the administrators was to learn from their experiences with the system and establish a baseline for expectations with regards to user behavior. These interviews included a selection of individuals in managerial and operational roles from the GDACS Secretariat and service branches. The administrative interviews took place between February and June 2015 in person and over the phone. User interviews were carried out over the phone or Skype throughout June 2015. Eight individuals were randomly drawn from the entire population of system users. Because we wanted to gauge the experience of the users over the lifetime of the system, we controlled the sample for users that had been registered prior to 2004 and that were currently active (i.e. had logged in and contributed comments to the Virtual OSOCC within the last year). The interview guide reflected on the history of the users and the Virtual OSOCC; user friendliness of the Virtual OSOCC; the purposes and overall performance of the system; different actors using the Virtual OSOCC; experiences particularly related to changes in the system and on the theme of information sharing. These interviews were transcribed and data analyzed by categorizing according to the questions asked. The important issues highlighted in the interviews are presented in the later parts of the paper. As a further supplement to our quantitative findings, we conducted an online survey among the group of users active during the last 5 years. This group consists of around 5,000 users out of whom 1,044 (21 percent) completed the survey. An important aspect of the survey design is that this was done after analyzing the quantitative user data and the semi-structured interviews to ensure consistency and increase relevance between the different data collection methods. For instance, the user survey includes a section on each of the main changes to the Virtual OSOCC with questions regarding the specific change and how this change affected the ease-ofuse and usefulness of the system. These questions were formulated as statements following the Likert scale approach to ordered responses. Descriptive statistics regarding the characteristics of the users surveyed are presented in Table 1. The users are predominantly men: 85 percent of the respondents are male and the Table 1. Descriptive statistics: User survey. Mean
Std. Dev.
0.852
0.356
43.198
9.425
0.672
0.47
Attachments
0.297
0.457
Comments
0.392
0.488
Relief teams
0.441
0.497
Relief items
0.169
0.375
Maps
0.201
0.401
Situation reports
0.332
0.471
Gender (male = 1) Age (years) Registered to receive GDACS alerts What type of information have you contributed to the VO?
Observations: 1,044 doi:10.1371/journal.pone.0161783.t001
PLOS ONE | DOI:10.1371/journal.pone.0161783 September 1, 2016
8 / 20
Technology and Information Sharing in Disaster Relief
median respondent is 43 years old. As expected, the descriptive statistics paint a picture typical of active users. Only 4 percent of the users surveyed state that they never access the Virtual OSOCC, while two-thirds access the system both during and between disaster events. Out of these, some 47 and 45 percent visit the site daily or hourly within the first week of the disaster, respectively. Regarding the type of contributions made to the system by users, the largest share of users have contributed to the system by using the relief team feature, while the second and third most important type of contribution are user comments and situation reports. In the next section we outline our strategy of exploiting the user data taken from the activity-log of the Virtual OSOCC to assess how external factors influence user behavior. We then go on to present first the quantitative and then the qualitative results before concluding.
Estimation strategy In order to test quantitatively the effect of external factors on user behavior, the optimal scenario is to compare actual and counterfactual outcomes (that is, what would have happened to user behavior in the absence of the updates to the system). In other words, one would ideally like to compare how the same users would have behaved with and without the changes to the technology. However, this is not possible since at a given point in time a user cannot have two simultaneous existences. The next best alternative is to compare outcomes of the users exposed to the change to those of a comparison group of unexposed users. Yet, since there exists no comparable technology to the Virtual OSOCC and it is not possible to limit a system intervention to a subset of the users, this approach is not viable. Rather we compare ex post behavior for users with data on their behavior before the system intervention. This approach is also referred to as the reflexive method of impact, where the users’ behavior before the intervention functions as a comparison or control outcome. This method is particularly relevant for the present study as the entire population of users is affected by the system interventions, in which case there is no scope for an external control group. Moreover, since the behavior of the users is observed over the entire life period of the system, structural changes in behavior can be tested for [23]. The main threat to the reflexive method is the presence of other time-varying (external) factors. For instance, imagine that right after the introduction of the new relief teams feature, users of the Virtual OSOCC start announcing a larger share of the actual number of teams deployed in the system. Although this increase in the number of relief teams announced may be due to the system change (through awareness of the usefulness of reporting team deployment), it may also be because the first events following the system change take place in countries considered to be particularly vulnerable or because certain types of disasters tend to generate more activity. To distinguish these effects it is therefore important to account for other factors explaining user behavior. The factors that are likely to affect information sharing can largely be grouped into either disaster specific characteristics or characteristics of the country hit by the disaster. Disaster specific characteristics include the type and severity of the disaster. We account for disaster type as the nature of assistance needed in the immediate aftermath of the disaster depends heavily on the type of the disaster. For instance, earthquakes in urban areas normally require more search and rescue teams compared to a flood. Additionally, more severe disasters are likely to spark more activity in the Virtual OSOCC as the need for assistance generally increases with the severity of the disaster. To account for the severity of disasters we combine the Virtual OSOCC activity log with data from the Emergency Events Database (EM-DAT) on the number of deaths caused by each disaster. A number of recoded disaster events in the Virtual OSOCC are not defined as a disaster in the EM-DAT database. For a disaster to be entered
PLOS ONE | DOI:10.1371/journal.pone.0161783 September 1, 2016
9 / 20
Technology and Information Sharing in Disaster Relief
into the EM-DAT database at least one of the following criteria must be fulfilled: (i) Ten or more people reported killed, (ii) Hundred or more people reported affected, (iii) Declaration of a state of emergency, (iv) call for international assistance. For these cases (20 observations), we complement the information from the database with other data sources, often summarized by Wikipedia. On average, the number of deaths in a disaster event in our sample is 4,406 people. Country specific characteristics include the vulnerability of affected countries and their level of democratization. The vulnerability of the affected country is likely to determine the involvement of the international community and thereby the level and type of activity observed in the Virtual OSOCC. We measure a country’s vulnerability by their Gross National Income (GNI) per capita using the World Bank Indicators. To ensure comparability across countries and years, GNI per capita is converted to a common currency at purchasing power parity. The mean GNI per capita of affected countries is 5,790—well below the global average of about 13,000 in 2010. We further account for the level of democratization since relief actions in some cases has been hindered by the lack of approval by the affected country’s government to enter the country in relation to rescue operations or distribution of water, food and other necessities; for instance during the Cyclone Nargis in Burma in 2008 where the military regime denied workers access to affected areas. The level of democratization is measured using the Polity IV Annual Time-Series database. The data categorize democratic and autocratic patterns of authority and regime changes in all independent countries with total population greater than 500,000 in 2014. For countries smaller than 500,000 inhabitants we use information collected from other sources to categorize the level of democracy. For disasters that hit more than one country we take the population weighted average of the affected countries GNI per capita and democratization index, respectively. The regression that we seek to estimate by OLS in order to evaluate the impact from external factors on user behavior in the Virtual OSOCC can be written as follows: yjt ¼ a þ ychanget þ b1 Djt þ b2 Xit þ gt þ d þ mjit
ð1Þ
where yjt is a vector of outcome measures for disaster j at time t, and the variable changet is an indicator variable for the change made to the Virtual OSOCC. As previously mentioned, the changes we consider in our baseline specification are: i) Re-design (2003), ii) introduction of GDACS alerts (2004), and iii) the introduction of four features: relief teams (2006), relief items (2006), maps (2004), and attachments (2005). The vectors Djt and Xit contain disaster and country specific characteristics, respectively. We also include region specific fixed effects (δ) as well as time fixed effects (γt). We define the regions based on the United Nations’ five geopolitical regional groups: the African Group, the Asian-Pacific Group, the Eastern European Group, the Latin American and Caribbean Group, and finally the Western European and Others Group. Time fixed effects are included in order to account for increasing user activity in the Virtual OSOCC over time. Finally, μjit is the error term containing all unobserved characteristics. We consider nine outcome measures that all approximate different aspects of user activity in the Virtual OSOCC. These outcomes can broadly be categorized into three groups: outcomes related to the overall use of the Virtual OSOCC, outcomes related to user behavior, and outcomes related to specific types of information. The first group of outcomes includes (i) the unique number of active user counts by recorded disaster, and (ii) the number of different organizations/institutions active by disaster. These outcomes will allow us to investigate how the number and type of users are related to the type and severity of the disaster, the vulnerability of the country hit and the type of user activity in the system. The user related outcomes include (iii) the number of comments added during the first day following an event creation,
PLOS ONE | DOI:10.1371/journal.pone.0161783 September 1, 2016
10 / 20
Technology and Information Sharing in Disaster Relief
(iv) the total number of comments created by disaster, and (v) the number of user comments by comment type and disaster. The third group of outcomes relate to specific pieces of information. These are contained in either comments or the different system features introduced in the system. For instance, we investigate whether the number of comments related to relief teams increased after the relief team feature was introduced. Specifically, the third group of outcomes includes the number of user-specific comments about (vi) relief teams, (vii) relief items, (viii) maps, and (viiii) attachments. We finally go even more into detail with subcategories of the relief teams feature.
Quantitative Results Based on a modified version of the TAM framework, we quantitatively test how user behavior related to information sharing changes with technological updates linked to external factors outside the control of the individual user. For this, we start by estimating Eq (1) using the Virtual OSOCC activity log. Results are presented in Table 2. Four dependent variables (yjt) are considered: unique user count, number of users active in disaster event, total number of comments, and the number of comments on the day the disaster event was created. We find that the system re-design in 2003 positively affect the number of users that commented in the system within the first 24 hours after the disaster event was activated in the Virtual OCOSS (column 1). Surprisingly, there is no detectable increase in the number of users that respond within a day compared to before the GDACS alerts system was introduced. This should be seen in relation to our user survey where 67 percent state that they are registered to receive GDACS alerts, and out of these around 75 percent agree that the alerts system helps reduce the time it takes to respond to a disaster event. Besides this, we find that the introduction of GDACS alerts is the system change that has generated the most significant increase in user activity in terms of number of users and comments (columns 1-3). The relief teams and the attachment features both seem to have generated a hike in the number of unique users (column 1). Also, we do not observe an expected decrease in the number of comments in column 3 following the introduction of these features which allow easier information sharing. In other words, although users are now able to share information on for instance relief team deployment by clicking a button instead of writing a comment, they are still active in the debate sections. In contrast, the relief item feature did not increase the number of unique users contributing to a given disaster event (column 1), but did, in line with expectations, decrease the number of comments concerning information related to relief items (column 3). Looking at the control variables, we find that disasters with more fatalities generate more comments and hence, information sharing. The same goes for disasters that happen in countries with a higher per capita GNI, although this relationship varies across information types. The level of democracy—as measured by the Polity IV index—does not seem to have any stand-alone effect on the amount of information shared. Interestingly, user activity is independent of the region in which the event happened (results not reported). Finally, we find that floods and storms/typhoons/cyclones generate less user activity compared to the earthquakes (i.e. the left out category). This is not surprising as earthquakes were initially the event type the system was designed to address. We now change the focus to look at outcomes related to the actual amount of information of different types shared in the system. Since we have categorized all the comments in the database according to the type of information they contain prior to the introduction of specific features, we are able to investigate how different pieces of information types are affected by the
PLOS ONE | DOI:10.1371/journal.pone.0161783 September 1, 2016
11 / 20
Technology and Information Sharing in Disaster Relief
Table 2. Baseline results.
Redesign of VO (= 1) GDACS alerts (= 1) Relief teams (= 1) Relief items (= 1) Attachments (= 1) No. of deaths (log) GNI per capita (log) Democratization Flood Storm/typhoons/cyclones Constant
(1)
(2)
(3)
(4)
Timespan
Unique user count
No. of inst.
No. of comments
1.342***
-0.019
0.184
0.327
(0.449)
(0.114)
(0.252)
(0.277)
0.417
0.400**
0.442*
0.532**
(0.378)
(0.177)
(0.244)
(0.257)
0.520
2.220***
0.160
1.107
(1.021)
(0.470)
(0.947)
(1.312)
-0.119
0.008
-0.783**
-1.492**
(0.543)
(0.278)
(0.394)
(0.655)
-0.007
0.665**
0.445
0.409
(0.653)
(0.306)
(0.557)
(0.697)
0.124***
0.109***
0.220***
0.314***
(0.032)
(0.017)
(0.022)
(0.028)
0.020
0.049
0.068
0.112
(0.087)
(0.034)
(0.066)
(0.076)
-0.007
0.011
0.023*
0.043***
(0.019)
(0.008)
(0.013)
(0.016)
-1.279***
-0.219***
-0.454***
-0.322*
(0.184)
(0.074)
(0.143)
(0.184)
-1.326***
-0.259***
-0.503***
-0.227
(0.179)
(0.084)
(0.143)
(0.174)
0.157
2.646***
0.277
0.608
(0.629)
(0.237)
(0.454)
(0.537)
Region dummies
Yes
Yes
Yes
Yes
Year dummies
Yes
Yes
Yes
Yes
Observations
160
215
215
215
Robust standard errors in parentheses. *** p
