Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

A Large-scale Benchmark on Geological Fault Delineation Models: Domain Shift, Training Dynamics, Generalizability, Evaluation and Inferential Behavior

Jorge Quesada, Chen Zhou, Prithwijit Chowdhury, Mohammad Alotaibi, Ahmad Mustafa, Yusufjon Kumakov, Mohit Prabhushankar, Ghassan AlRegib

TL;DR

This paper addresses the generalization problem in seismic fault delineation under domain shift by delivering the first large-scale, open benchmarking framework. It evaluates 200+ training configurations across 3 heterogeneous datasets (FaultSeg3D, CRACKS, Thebe) and 8 architectures, examining pretraining, fine-tuning, joint training, and domain-adaptation strategies, complemented by fault-characteristic metric analysis. Key findings show that larger models like SegFormer often generalize better when source-target domains are aligned, while fine-tuning alone can cause catastrophic forgetting under strong shifts; domain adaptation helps for large shifts but can hurt when domains are similar. The work provides practical guidelines and an open-source codebase to support reproducible cross-domain evaluation and to drive the development of more interpretable, geometry-aware fault delineation in real-world seismic interpretation workflows.

Abstract

Machine learning has taken a critical role in seismic interpretation workflows, especially in fault delineation tasks. However, despite the recent proliferation of pretrained models and synthetic datasets, the field still lacks a systematic understanding of the generalizability limits of these models across seismic data representing diverse geologic, acquisition and processing settings. Distributional shifts between data sources, limitations in fine-tuning strategies and labeled data accessibility, and inconsistent evaluation protocols all remain major roadblocks to deploying reliable models in real-world exploration. In this paper, we present the first large-scale benchmarking study explicitly designed to provide guidelines for domain shift strategies in seismic interpretation. Our benchmark spans over 200 combinations of model architectures, datasets and training strategies, across three datasets (synthetic and real) including FaultSeg3D, CRACKS, and Thebe. We systematically assess pretraining, fine-tuning, and joint training under varying domain shifts. Our analysis shows that common fine-tuning practices can lead to catastrophic forgetting, especially when source and target datasets are disjoint, and that larger models such as Segformer are more robust than smaller architectures. We also find that domain adaptation methods outperform fine-tuning when shifts are large, yet underperform when domains are similar. Finally, we complement segmentation metrics with a novel analysis based on fault characteristic descriptors, revealing how models absorb structural biases from training datasets. Overall, we establish a robust experimental baseline that provides insights into tradeoffs in current fault delineation workflows and highlights directions for building more generalizable and interpretable models.

A Large-scale Benchmark on Geological Fault Delineation Models: Domain Shift, Training Dynamics, Generalizability, Evaluation and Inferential Behavior

TL;DR

This paper addresses the generalization problem in seismic fault delineation under domain shift by delivering the first large-scale, open benchmarking framework. It evaluates 200+ training configurations across 3 heterogeneous datasets (FaultSeg3D, CRACKS, Thebe) and 8 architectures, examining pretraining, fine-tuning, joint training, and domain-adaptation strategies, complemented by fault-characteristic metric analysis. Key findings show that larger models like SegFormer often generalize better when source-target domains are aligned, while fine-tuning alone can cause catastrophic forgetting under strong shifts; domain adaptation helps for large shifts but can hurt when domains are similar. The work provides practical guidelines and an open-source codebase to support reproducible cross-domain evaluation and to drive the development of more interpretable, geometry-aware fault delineation in real-world seismic interpretation workflows.

Abstract

Machine learning has taken a critical role in seismic interpretation workflows, especially in fault delineation tasks. However, despite the recent proliferation of pretrained models and synthetic datasets, the field still lacks a systematic understanding of the generalizability limits of these models across seismic data representing diverse geologic, acquisition and processing settings. Distributional shifts between data sources, limitations in fine-tuning strategies and labeled data accessibility, and inconsistent evaluation protocols all remain major roadblocks to deploying reliable models in real-world exploration. In this paper, we present the first large-scale benchmarking study explicitly designed to provide guidelines for domain shift strategies in seismic interpretation. Our benchmark spans over 200 combinations of model architectures, datasets and training strategies, across three datasets (synthetic and real) including FaultSeg3D, CRACKS, and Thebe. We systematically assess pretraining, fine-tuning, and joint training under varying domain shifts. Our analysis shows that common fine-tuning practices can lead to catastrophic forgetting, especially when source and target datasets are disjoint, and that larger models such as Segformer are more robust than smaller architectures. We also find that domain adaptation methods outperform fine-tuning when shifts are large, yet underperform when domains are similar. Finally, we complement segmentation metrics with a novel analysis based on fault characteristic descriptors, revealing how models absorb structural biases from training datasets. Overall, we establish a robust experimental baseline that provides insights into tradeoffs in current fault delineation workflows and highlights directions for building more generalizable and interpretable models.
Paper Structure (32 sections, 13 equations, 15 figures, 9 tables)

This paper contains 32 sections, 13 equations, 15 figures, 9 tables.

Table of Contents

  1. Introduction
  2. Related Work
  3. Data Preparation and Preprocessing
  4. Training Dynamics and Setups
  5. Generalization and Transferability
  6. Inferential Behavior
  7. Performance Evaluation
  8. Fault Characteristics Metrics
  9. Fault Delineation Datasets
  10. Experimental Setup
  11. Data Preparation
  12. Standardizing Fault Annotations using Morphological Operations
  13. Training and Test Splits
  14. Model setup
  15. Evaluation Setup
  16. ...and 17 more sections

Figures (15)

  • Figure 1: Typical DL-assisted seismic interpretation workflow
  • Figure 2: Finetuning example from a real dataset (D1) and a synthetic dataset (D2) to a target real dataset
  • Figure 3: Visuals from three datasets: (\ref{['fig:datase-sub1']}) synthesized faults in FaultSeg3D, (\ref{['fig:datase-sub2']}) expert labels in CRACKS, and (\ref{['fig:datase-sub3']}) expert labels in Thebe.
  • Figure 4: Charaterization of the three considered datasets using fault-oriented metrics.
  • Figure 5: The block diagram of our experimental setup.
  • ...and 10 more figures