Expect the Unexpected: FailSafe Long Context QA for Finance

TL;DR

FailSafeQA introduces a long-context financial QA benchmark to evaluate LLM robustness and context grounding under user-interface perturbations, using 10-K filings as primary sources. The dataset combines query and context perturbations (misspellings, incomplete and out-of-domain queries; missing, OCR-ed, and irrelevant contexts) and uses an LLM-as-a-Judge framework to quantify metrics such as Answer Relevance, Answer Compliance, Robustness, and Context Grounding, culminating in a LLMC$_{\beta}$ trade-off score. Results show substantial room for improvement: even the best-performing models struggle with long-context hallucinations, especially under OCR and missing-context conditions, while some models excel at grounding but falter on robustness, highlighting the need for fail-safe design in finance applications. The work provides reproducible benchmarks and suggests that future improvements should prioritize dependable grounding and safe refusal when information is unavailable, with potential extensions to additional domains and multi-source aggregation.

Abstract

We propose a new long-context financial benchmark, FailSafeQA, designed to test the robustness and context-awareness of LLMs against six variations in human-interface interactions in LLM-based query-answer systems within finance. We concentrate on two case studies: Query Failure and Context Failure. In the Query Failure scenario, we perturb the original query to vary in domain expertise, completeness, and linguistic accuracy. In the Context Failure case, we simulate the uploads of degraded, irrelevant, and empty documents. We employ the LLM-as-a-Judge methodology with Qwen2.5-72B-Instruct and use fine-grained rating criteria to define and calculate Robustness, Context Grounding, and Compliance scores for 24 off-the-shelf models. The results suggest that although some models excel at mitigating input perturbations, they must balance robust answering with the ability to refrain from hallucinating. Notably, Palmyra-Fin-128k-Instruct, recognized as the most compliant model, maintained strong baseline performance but encountered challenges in sustaining robust predictions in 17% of test cases. On the other hand, the most robust model, OpenAI o3-mini, fabricated information in 41% of tested cases. The results demonstrate that even high-performing models have significant room for improvement and highlight the role of FailSafeQA as a tool for developing LLMs optimized for dependability in financial applications. The dataset is available at: https://huggingface.co/datasets/Writer/FailSafeQA

Figures (6)

• Figure 1: FailSafeQA: Robustness and Context Grounding Evaluation We evaluate the resilience of an LLM-based QA system in two case studies: Query Failure and Context Failure. In the Query Failure scenario, we perturb the original query into three variants: containing spelling errors (Misspelled Query), query-term form (Incomplete Query), rephrased to exclude in-domain terminology (Out-of-Domain Query). In the Context Failure case, we assume users can either fail to upload the document (Missing Context) , use degraged quality documents due to OCR (OCRed Context) or upload a document irrelevant to the query (Irrelevant Context). Robustness involves maintaining consistent model performance across perturbations (A)-(C) and (E), which preserve the intended meaning, while Context Grounding involves preventing hallucinations in scenarios (D) and (F).
• Figure 2: The Dataset Analysis of root verbs and their direct objects from the first sentence of each normalized query shows the top 20 verbs and their top five direct objects. This distribution can be used as a proxy measure for the diversity of tasks in the dataset, with 83.0% related to question answering (QA) and 17.0% involving text generation (TG).
• Figure 3: Answer Relevance Classes We evaluate two scenarios in our benchmark: when models should provide an answer (ANSWER QUERY) and when they must decline to answer (REFUSE QUERY) due to lack of relevant context. Our findings reveal that all the tested models are more adept at offering suitable answers than providing a justified refusal in situations where the context lacks sufficient information. Among all models evaluated, Palmyra-Fin-128k-Instruct demonstrates the most effective balance between these capabilities.
• Figure 4: Robustness and Compliance (Left) All models lose with respect to the baseline when input perturbations are applied. The biggest drop is observed for Out-Of-Domain and OCR context perturbations. Among the $24$ tested models, OpenAI o3-mini is the most robust. (Right) Reasoning models like OpenAI-o1/o3-mini and the DeepSeek-R1 series reach scores up to $0.59$, while Qwen models consistently surpass $0.60$. Palmyra-Fin-128k-Instruct excels with the highest Context Grounding score of $0.80$.
• Figure 5: Robustness vs. Query Type. (Left) Across all models, the decrease in robustness is more prominent in text generation (TG) than in question-answering (QA) tasks. (Right) Similar statement also holds true for Context Grounding - when a model is asked to generate text (e.g., a blog post), it is more likely to ignore the lack of relevant information and fabricate details. For almost all models, it is easier to refuse to answer based on a wrong document (irrelevant context) than to deal with empty context (e.g., due to a failed document upload).