Evaluating LLM Understanding via Structured Tabular Decision Simulations

TL;DR

STaDS addresses the gap between predictive accuracy and genuine understanding in LLMs by evaluating three complementary dimensions: instruction comprehension, knowledge-based prediction, and reliance on correct decision factors, using a suite of 15 real-world tabular datasets. It introduces a formal metric framework including Comprehension Fidelity ($ Len ext{-}F1 $, UnkLbl%), Predictive Competence (Accuracy, Macro-F1, PenAcc), and Decision Faithfulness (SelfAtt@k, LAO, Self-Faith, $ ho$, $\sigma_{LAO}$) to assess global behavior rather than single-instance correctness. Across 9 frontier models, results reveal that high accuracy often coexists with global faithfulness violations, and demonstrated in-context learning improves performance but does not guarantee stable, domain-grounded decision factors. The work positions STaDS as a reproducible, extensible protocol bridging accuracy, interpretability, and explainability, and it advocates for moving beyond local reasoning traces to evaluate true understanding in LLMs across diverse domains.

Abstract

Large language models (LLMs) often achieve impressive predictive accuracy, yet correctness alone does not imply genuine understanding. True LLM understanding, analogous to human expertise, requires making consistent, well-founded decisions across multiple instances and diverse domains, relying on relevant and domain-grounded decision factors. We introduce Structured Tabular Decision Simulations (STaDS), a suite of expert-like decision settings that evaluate LLMs as if they were professionals undertaking structured decision ``exams''. In this context, understanding is defined as the ability to identify and rely on the correct decision factors, features that determine outcomes within a domain. STaDS jointly assesses understanding through: (i) question and instruction comprehension, (ii) knowledge-based prediction, and (iii) reliance on relevant decision factors. By analyzing 9 frontier LLMs across 15 diverse decision settings, we find that (a) most models struggle to achieve consistently strong accuracy across diverse domains; (b) models can be accurate yet globally unfaithful, and there are frequent mismatches between stated rationales and factors driving predictions. Our findings highlight the need for global-level understanding evaluation protocols and advocate for novel frameworks that go beyond accuracy to enhance LLMs' understanding ability.

Figures (15)

• Figure 1: This diagram illustrates how the STaDS protocol simulates expert decision-making processes in structured tabular decision settings. The protocol evaluates LLMs' understanding ability through three key dimensions: (1) Question and Instruction Comprehension, assessing task interpretation and output adherence; (2) Knowledge-based Prediction, evaluating the model's application of domain knowledge for accurate predictions; and (3) Reliance on the Right Decision Factors, determining whether predictions align with the factors the model claims to rely on. The diagram depicts how these dimensions together form a principled basis for understanding and evaluating LLMs.
• Figure 2: The diagram illustrates the process flow from tabular data preprocessing across various domains to prompt construction for LLMs. The model is tasked with predicting labels based on structured input rows and is evaluated along three key dimensions of understanding. Question and Instruction Comprehension is measured by Len-F1, UnkLbl%, and $\Delta_{\text{PenAcc}}$, Predictive Competence is quantified through Accuracy, Macro-F1, and PenAcc, and Reliance on the Right Decision Factors is measured by SelfAtt@k, Self-Decision Faithfulness and LAO Magnitude.
• Figure 3: Box plots illustrate the distribution of $\Delta_\text{acc} = \mathrm{Acc} - \mathrm{PenAcc}$ for each model across all benchmark datasets. Blue and orange correspond to few-shot and zero-shot settings, respectively. Frontier models cluster near zero $\Delta_\text{acc}$, while several open-source checkpoints incur format penalties, especially in few-shot setting, indicating heightened prompt sensitivity.
• Figure 4: Spider plots of Penalized Accuracy ($\alpha = 0.5, \beta = 0.5$) across models and datasets in (a) zero-shot and (b) few-shot settings. Each axis is a model; each colored trace is a dataset. Higher values indicate stronger accuracy and instruction-following. Few-shot generally inflates the polygons (with higher PenAcc) across datasets, with Gemini-2.5-Pro showing the most uniform gains.
• Figure 5: Heatmap of LAO performance ($\Delta_{\text{LAO}}$) for each feature (columns) and LLM (rows). Darker blue indicates a larger performance loss when the feature is removed (higher importance); red indicates a slight performance gain or negligible reliance. A few features dominate reliance for certain models (deep blue), while others spread reliance diffusely, consistent with their $\sigma_{\text{LAO}}$.

Theorems & Definitions (1)

• Remark 5.1