Landslide Detection in Real-Time Social Media Image Streams

TL;DR

Global landslide data scarcity motivates automatic detection in social media image streams using AI. The authors assemble a large expert-labeled dataset (11,737 images) from multiple sources and perform extensive CNN-based transfer learning to detect landslides in images. The best configuration (ResNet50, Adam, learning rate 1e-4, weight decay 1e-3, no balancing) achieves validation F1 of 0.805 and accuracy of 0.913, with a drop on the test set to F1 0.701 and accuracy 0.870, analyzed via CAMs and t-SNE visualizations. The work outlines a real-time deployment path—Tweet and Image collectors, image classifier, geolocation, and dashboards—aiming to augment global landslide susceptibility maps and emergency response.

Abstract

Lack of global data inventories obstructs scientific modeling of and response to landslide hazards which are oftentimes deadly and costly. To remedy this limitation, new approaches suggest solutions based on citizen science that requires active participation. However, as a non-traditional data source, social media has been increasingly used in many disaster response and management studies in recent years. Inspired by this trend, we propose to capitalize on social media data to mine landslide-related information automatically with the help of artificial intelligence (AI) techniques. Specifically, we develop a state-of-the-art computer vision model to detect landslides in social media image streams in real time. To that end, we create a large landslide image dataset labeled by experts and conduct extensive model training experiments. The experimental results indicate that the proposed model can be deployed in an online fashion to support global landslide susceptibility maps and emergency response.

Figures (3)

• Figure 1: Example images from the dataset.
• Figure 2: Visualization of the feature embeddings before/after model finetuning
• Figure 3: Class activation map visualizations of the model predictions on the test set.