Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Introducing Axlerod: An LLM-based Chatbot for Assisting Independent Insurance Agents

Adam Bradley, John Hastings, Khandaker Mamun Ahmed

TL;DR

The paper tackles the need for efficient, accurate policy information retrieval in insurance by introducing Axlerod, an LLM-based agent-assistive chatbot. It combines retrieval-augmented generation with structured tool integration to query a large, multi-source data stack of policies and documents, implemented via a lightweight middleware and the Smoltalk framework atop Google's Gemini 2.5 Pro. Key contributions include an end-to-end agent-support system evaluated in collaboration with Safety Insurance, with an overall accuracy of $93.18\%$ and a $2.42$-second average time saving per task, alongside a detailed cost assessment and a representative interaction illustrating iterative clarification. The work demonstrates the feasibility and impact of enterprise-grade AI in insurtech for agent productivity, while emphasizing Safety, transparency, and human-in-the-loop oversight to mitigate biases and errors.

Abstract

The insurance industry is undergoing a paradigm shift through the adoption of artificial intelligence (AI) technologies, particularly in the realm of intelligent conversational agents. Chatbots have evolved into sophisticated AI-driven systems capable of automating complex workflows, including policy recommendation and claims triage, while simultaneously enabling dynamic, context-aware user engagement. This paper presents the design, implementation, and empirical evaluation of Axlerod, an AI-powered conversational interface designed to improve the operational efficiency of independent insurance agents. Leveraging natural language processing (NLP), retrieval-augmented generation (RAG), and domain-specific knowledge integration, Axlerod demonstrates robust capabilities in parsing user intent, accessing structured policy databases, and delivering real-time, contextually relevant responses. Experimental results underscore Axlerod's effectiveness, achieving an overall accuracy of 93.18% in policy retrieval tasks while reducing the average search time by 2.42 seconds. This work contributes to the growing body of research on enterprise-grade AI applications in insurtech, with a particular focus on agent-assistive rather than consumer-facing architectures.

Introducing Axlerod: An LLM-based Chatbot for Assisting Independent Insurance Agents

TL;DR

The paper tackles the need for efficient, accurate policy information retrieval in insurance by introducing Axlerod, an LLM-based agent-assistive chatbot. It combines retrieval-augmented generation with structured tool integration to query a large, multi-source data stack of policies and documents, implemented via a lightweight middleware and the Smoltalk framework atop Google's Gemini 2.5 Pro. Key contributions include an end-to-end agent-support system evaluated in collaboration with Safety Insurance, with an overall accuracy of and a -second average time saving per task, alongside a detailed cost assessment and a representative interaction illustrating iterative clarification. The work demonstrates the feasibility and impact of enterprise-grade AI in insurtech for agent productivity, while emphasizing Safety, transparency, and human-in-the-loop oversight to mitigate biases and errors.

Abstract

The insurance industry is undergoing a paradigm shift through the adoption of artificial intelligence (AI) technologies, particularly in the realm of intelligent conversational agents. Chatbots have evolved into sophisticated AI-driven systems capable of automating complex workflows, including policy recommendation and claims triage, while simultaneously enabling dynamic, context-aware user engagement. This paper presents the design, implementation, and empirical evaluation of Axlerod, an AI-powered conversational interface designed to improve the operational efficiency of independent insurance agents. Leveraging natural language processing (NLP), retrieval-augmented generation (RAG), and domain-specific knowledge integration, Axlerod demonstrates robust capabilities in parsing user intent, accessing structured policy databases, and delivering real-time, contextually relevant responses. Experimental results underscore Axlerod's effectiveness, achieving an overall accuracy of 93.18% in policy retrieval tasks while reducing the average search time by 2.42 seconds. This work contributes to the growing body of research on enterprise-grade AI applications in insurtech, with a particular focus on agent-assistive rather than consumer-facing architectures.
Paper Structure (13 sections, 2 figures, 1 table)

This paper contains 13 sections, 2 figures, 1 table.

Table of Contents

  1. Introduction
  2. Related Work
  3. System Design and Implementation
  4. Agent Design
  5. Data Description
  6. Tools and Technologies
  7. Experimental Analysis
  8. Accuracy
  9. Evaluation of Time Savings
  10. Cost
  11. Example Conversation
  12. Future Work
  13. Conclusion

Figures (2)

  • Figure 1: Overall architecture of Axlerod. The system connects an LLM (Google Gemini 2.5 Pro) with Safety Insurance’s internal data sources through a lightweight middleware layer. Three primary tools: policy detail, policy search, and documentation search are exposed to the LLM for structured retrieval.
  • Figure 2: The accuracy of query responses by the developed AI chatbot.