Training an Emergency-Response Image Classifier on Signal Data

arXiv (Cornell University), 2021

Images shared on social media help crisis managers gain situational awareness and assess incurred damages, among other response tasks. As the volume and velocity of such content are typically high, real-time image classification has become an urgent need for a faster disaster response. Recent advances in computer vision and deep neural networks have enabled the development of models for real-time image classification for a number of tasks, including detecting crisis incidents, filtering irrelevant images, classifying images into specific humanitarian categories, and assessing the severity of the damage. To develop robust real-time models, it is necessary to understand the capability of the publicly available pre-trained models for these tasks, which remains to be under-explored in the crisis informatics literature. In this study, we address such limitations by investigating ten different network architectures for four different tasks using the largest publicly available datasets for these tasks. We also explore various data augmentation strategies, semi-supervised techniques, and a multitask learning setup. In our extensive experiments, we achieve promising results.

Progress in Disaster Science, 2019

Using social media during natural disasters has become commonplace globally. In the U.S., public social media platforms are often a go-to because people believe: the 9-1-1 system becomes overloaded during emergencies and that first responders will see their posts. While social media requests may help save lives, these posts are difficult to find because there is more noise on public social media than clear signals of who needs help. This study compares human-coded images posted during 2017's Hurricane Harvey to machine-learned 'deep learning' classification methods. Our framework for feature extraction uses the VGG-16 convolutional neural network/multilayer perceptron classifiers for classifying the urgency and time period for a given image. We find that our qualitative results showcase that unique disaster experiences are not always captured through machine-learned methods. These methods work together to parse through the high levels of non-relevant content on social media to find relevant content and requests.

IEEE Access, 2021

Social media communication serves as an integral part of the crisis response following a mass emergency (disaster) event. Regardless of the kind of disaster event, whether it is a hurricane, a flood, an earthquake or a man-made disaster event like a riot or a terrorist attack, social media platforms like Facebook, Twitter etc. have proven to be a powerful facilitator of communication and coordination between disaster victims and other communities. Consequently, several research articles have been published on social media utilization for disaster response. Many of those recent research articles discuss automated machine learning approaches to extract disaster indicating posts, useful for coordination from various social media posts. Despite this, there is a scarcity of comprehensive review of all the major research works pertaining to the utilization of machine learning approaches for disaster response using social media posts. Thus, this study reviews academic research articles in the domain and classifies them across three disaster phase dimensionsearly warning and event detection, post-disaster coordination and response, damage assessment. This review would help researchers in choosing further research topics pertaining to automated approaches for actionable information classification and disaster coordination and would help the emergency teams to make wellinformed decisions in disaster situations.

2021

Images shared on social media help crisis managers in terms of gaining situational awareness and assessing incurred damages, among other response tasks. As the volume and velocity of such content are really high, therefore, real-time image classification became an urgent need in order to take a faster response. Recent advances in computer vision and deep neural networks have enabled the development of models for real-time image classification for a number of tasks, including detecting crisis incidents, filtering irrelevant images, classifying images into specific humanitarian categories, and assessing the severity of the damage. For developing real-time robust models, it is necessary to understand the capability of the publicly available pretrained models for these tasks. In the current state-of-art of crisis informatics, it is under-explored. In this study, we address such limitations. We investigate ten different architectures for four different tasks using the largest publicly av...

2024 International Conference on Information and Communication Technologies for Disaster Management (ICT-DM)

Social media's real-time nature has transformed it into a critical tool for disaster response, and for that this study explores the use of tweets for classifying disaster types and identifying humanitarian needs in the aftermath of various disaster events. We compare traditional machine learning models like Random Forest and Support Vector Machines with the deep learning technique, BERT. While BERT demonstrates promising results, a key finding lies in the performance of the voting classifier ensemble, a combination of traditional models. This ensemble achieves accuracy comparable to BERT and even surpasses it. Furthermore, the ensemble boasts exceptional training and inference speeds, making it ideal for real-time applications in disaster response scenarios. Our work investigates the continued value of traditional machine learning methods. By "dusting off" these models we can achieve competitive performance while maintaining computational efficiency. Ultimately, this study empowers humanitarian organizations to leverage the power of text classification for extracting crucial insights from social media data, leading to more effective and targeted responses in times of crisis.

2020 IEEE/ACM International Conference on Advances in Social Networks Analysis and Mining (ASONAM)

During a disaster event, images shared on social media helps crisis managers gain situational awareness and assess incurred damages, among other response tasks. Recent advances in computer vision and deep neural networks have enabled the development of models for real-time image classification for a number of tasks, including detecting crisis incidents, filtering irrelevant images, classifying images into specific humanitarian categories, and assessing the severity of damage. Despite several efforts, past works mainly suffer from limited resources (i.e., labeled images) available to train more robust deep learning models. In this study, we propose new datasets for disaster type detection, and informativeness classification, and damage severity assessment. Moreover, we relabel existing publicly available datasets for new tasks. We identify exact-and near-duplicates to form non-overlapping data splits, and finally consolidate them to create larger datasets. In our extensive experiments, we benchmark several state-of-the-art deep learning models and achieve promising results. We release our datasets and models publicly, aiming to provide proper baselines as well as to spur further research in the crisis informatics community.

The extensive use of social media platforms, especially during disasters, creates unique opportunities for humanitarian organizations to gain situational awareness as disaster unfolds. In addition to textual content, people post overwhelming amounts of imagery content on social networks within minutes of a disaster hit. Studies point to the importance of this online imagery content for emergency response. Despite recent advances in computer vision research, making sense of the imagery content in real-time during disasters remains a challenging task. One of the important challenges is that a large proportion of images shared on social media is redundant or irrelevant, which requires robust filtering mechanisms. Another important challenge is that images acquired after major disasters do not share the same characteristics as those in large-scale image collections with clean annotations of well-defined object categories such as house, car, airplane, cat, dog, etc., used traditionally in computer vision research. To tackle these challenges, we present a social media image processing pipeline that combines human and machine intelligence to perform two important tasks: (i) capturing and filtering of social media imagery content (i.e., real-time image streaming, de-duplication, and relevancy filtering), and (ii) actionable information extraction (i.e., damage severity assessment) as a core situational awareness task during an ongoing crisis event. Results obtained from extensive experiments on real-world crisis datasets demonstrate the significance of the proposed pipeline for optimal utilization of both human and machine computing resources.

Journal of the Association for Information Science and Technology

This article addresses the problem of detecting crisisrelated messages on social media, in order to improve the situational awareness of emergency services. Previous work focused on developing machine-learning classifiers restricted to specific disasters, such as storms or wildfires. We investigate for the first time methods to detect such messages where the type of the crisis is not known in advance, that is, the data are highly heterogeneous. Data heterogeneity causes significant difficulties for learning algorithms to generalize and accurately label incoming data. Our main contributions are as follows. First, we evaluate the extent of this problem in the context of disaster management, finding that the performance of traditional learners drops by up to 40% when trained and tested on heterogeneous data vis-á-vis homogeneous data. Then, in order to overcome data heterogeneity, we propose a new ensemble learning method, and found this to perform on a par with the Gradient Boosting and AdaBoost ensemble learners. The methods are studied on a benchmark data set comprising 26 disaster events and four classification problems: detection of relevant messages, informative messages, eyewitness reports, and topical classification of messages. Finally, in a case study, we evaluate the proposed methods on a real-world data set to assess its practical value.

2020

Flooding management requires collecting real-time onsite information widely and rapidly. As an emerging data source, social media demonstrates an advantage of providing in-time, rich data in the format of texts and photos and can be used to improve flooding situation awareness. The present study shows that social media data, with additional information processed by Artificial Intelligence (AI) techniques, can be effectively used to track flooding phase transition and locate emergency incidents. To track phase transition, we train a computer vision model that can classify images embedded in social media data into four categoriespreparedness, impact, response, and recovery-that can reflect the phases of disaster event development. To locate emergency incidents, we use a deep learning based natural language processing (NLP) model to recognize locations from textual content of tweets. The geographic coordinates of the recognized locations are assigned by searching through a dedicated local gazetteer rapidly compiled for the disaster affected region based on the GeoNames gazetteer and the US Census data. By combining image and text analysis, we filter the tweets that contain images of the "Impact" category and high-resolution locations to gain the most valuable situation information. We carry out a manual examination step to complement the automatic data processing and find that it can further strengthen the AI-processed results to support comprehensive situation awareness and to establish a passive hotline to inform rescue and search activities. The developed framework is applied to the flood of Hurricane Harvey in the Houston area.

Journal of Big Data, 2021

Social media postings are increasingly being used in modern days disaster management. Along with the textual information, the contexts and cues inherent in the images posted on social media play an important role in identifying appropriate emergency responses to a particular disaster. In this paper, we proposed a disaster taxonomy of emergency response and used the same taxonomy with an emergency response pipeline together with deep-learning-based image classification and object identification algorithms to automate the emergency response decision-making process. We used the card sorting method to validate the completeness and correctness of the disaster taxonomy. We also used VGG-16 and You Only Look Once (YOLO) algorithms to analyze disaster-related images and identify disaster types and relevant cues (such as objects that appeared in those images). Furthermore, using decision tables and applied analytic hierarchy processes (AHP), we aligned the intermediate outputs to map a disas...