Reasoning Inconsistencies and How to Mitigate Them in Deep Learning
Erik Arakelyan
TL;DR
This thesis investigates reasoning inconsistencies in deep learning across NLP, vision, and knowledge graphs, identifying three primary causes: opaque internal procedures, data imbalances, and task complexity. It introduces a cohesive framework comprising adversarial setups to quantify internal-process fragility, topic-guided sampling and synthetic data generation to mitigate data-driven biases, and two methods—CQD adaptation and FLARE—to optimize and faithfully reason over complex questions. The work presents six papers spanning semantics sensitivity in NLI, adversarial transferability of backbones, data-efficient stance detection, synthetic QA data for low-resource languages, data-efficient complex query answering on KG, and a faithful logic-aided reasoning framework, FLARE, that integrates planning, formalisation, and simulated search. Collectively, these contributions advance robustness, fairness, and interpretability of deep learning models across modalities, enabling more reliable, auditable reasoning in real-world applications. The results demonstrate meaningful performance gains and improved faithfulness on diverse tasks, highlighting practical implications for deploying trustworthy AI systems with transparent reasoning paths.
Abstract
The recent advancements in Deep Learning models and techniques have led to significant strides in performance across diverse tasks and modalities. However, while the overall capabilities of models show promising growth, our understanding of their internal reasoning processes remains limited, particularly concerning systematic inconsistencies or errors patterns of logical or inferential flaws. These inconsistencies may manifest as contradictory outputs, failure to generalize across similar tasks, or erroneous conclusions in specific contexts. Even detecting and measuring such reasoning discrepancies is challenging, as they may arise from opaque internal procedures, biases and imbalances in training data, or the inherent complexity of the task. Without effective methods to detect, measure, and mitigate these errors, there is a risk of deploying models that are biased, exploitable, or logically unreliable. This thesis aims to address these issues by producing novel methods for deep learning models that reason over knowledge graphs, natural language, and images. The thesis contributes two techniques for detecting and quantifying predictive inconsistencies originating from opaque internal procedures in natural language and image processing models. To mitigate inconsistencies from biases in training data, this thesis presents a data efficient sampling method to improve fairness and performance and a synthetic dataset generation approach in low resource scenarios. Finally, the thesis offers two techniques to optimize the models for complex reasoning tasks. These methods enhance model performance while allowing for more faithful and interpretable exploration and exploitation during inference. Critically, this thesis provides a comprehensive framework to improve the robustness, fairness, and interpretability of deep learning models across diverse tasks and modalities.