Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

ITLC at SemEval-2026 Task 11: Normalization and Deterministic Parsing for Formal Reasoning in LLMs

Wicaksono Leksono Muhamad, Joanito Agili Lopo, Tack Hwa Wong, Muhammad Ravi Shulthan Habibi, Samuel Cahyawijaya

TL;DR

This work introduces a novel method that reduces biases through explicit structural abstraction that transforms syllogisms into canonical logical representations and applies deterministic parsing to determine validity in large language models.

Abstract

Large language models suffer from content effects in reasoning tasks, particularly in multi-lingual contexts. We introduce a novel method that reduces these biases through explicit structural abstraction that transforms syllogisms into canonical logical representations and applies deterministic parsing to determine validity. Evaluated on the SemEval-2026 Task 11 multilingual benchmark, our approach achieves top-5 rankings across all subtasks while substantially reducing content effects and offering a competitive alternative to complex fine-tuning or activation-level interventions.

ITLC at SemEval-2026 Task 11: Normalization and Deterministic Parsing for Formal Reasoning in LLMs

TL;DR

This work introduces a novel method that reduces biases through explicit structural abstraction that transforms syllogisms into canonical logical representations and applies deterministic parsing to determine validity in large language models.

Abstract

Large language models suffer from content effects in reasoning tasks, particularly in multi-lingual contexts. We introduce a novel method that reduces these biases through explicit structural abstraction that transforms syllogisms into canonical logical representations and applies deterministic parsing to determine validity. Evaluated on the SemEval-2026 Task 11 multilingual benchmark, our approach achieves top-5 rankings across all subtasks while substantially reducing content effects and offering a competitive alternative to complex fine-tuning or activation-level interventions.
Paper Structure (43 sections, 4 equations, 7 figures, 20 tables, 2 algorithms)

This paper contains 43 sections, 4 equations, 7 figures, 20 tables, 2 algorithms.

Table of Contents

  1. Introduction
  2. Background
  3. Categorical Syllogisms
  4. Logical Structure and Terminology
  5. Mood, Figure, and Validity
  6. Trivial Validity
  7. System Overview
  8. Normalization
  9. Categorical Syllogism
  10. English Pivot Normalization
  11. Preposition Parsing
  12. Formal Validation
  13. Logical Validity
  14. Relevant Premises Identification
  15. Experimental Setup
  16. ...and 28 more sections

Figures (7)

  • Figure 1: The flowchart illustrates the example step by step the flow of the proposed system.
  • Figure 2: Content-effect reduction in English-only and Multilingual
  • Figure 3: LLM-only prompt for retrieve the validity and relevant premise directly
  • Figure 4: Norm prompt for normalize sentences into standard categorical form
  • Figure 5: EPN prompt for Subtask 3 for extract subject term
  • ...and 2 more figures