Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

From Policy to Logic for Efficient and Interpretable Coverage Assessment

Rhitabrat Pokharel, Hamid Hassanzadeh, Ameeta Agrawal

TL;DR

The paper tackles reliable, interpretable policy interpretation for medical CPT coverage by proposing a neuro-symbolic framework that couples a coverage-aware retriever with a PyKnow-based rule engine to surface governing policy language and generate auditable rationales, reducing reliance on expensive LLM inferences. It introduces a large expert-labeled retriever dataset and a contrastive ranking objective to select relevant policy passages, followed by automatic attribute extraction and rule generation that feed a symbolic inference engine. Key findings show a 44% reduction in inference cost and a 4.5% improvement in F1 when using the proposed hybrid approach, while maintaining human adjudication authority. The work demonstrates that combining targeted retrieval with symbolic reasoning yields scalable, interpretable, and cost-effective support for policy review in healthcare, addressing both reliability and efficiency concerns in real-world adjudication.

Abstract

Large Language Models (LLMs) have demonstrated strong capabilities in interpreting lengthy, complex legal and policy language. However, their reliability can be undermined by hallucinations and inconsistencies, particularly when analyzing subjective and nuanced documents. These challenges are especially critical in medical coverage policy review, where human experts must be able to rely on accurate information. In this paper, we present an approach designed to support human reviewers by making policy interpretation more efficient and interpretable. We introduce a methodology that pairs a coverage-aware retriever with symbolic rule-based reasoning to surface relevant policy language, organize it into explicit facts and rules, and generate auditable rationales. This hybrid system minimizes the number of LLM inferences required which reduces overall model cost. Notably, our approach achieves a 44% reduction in inference cost alongside a 4.5% improvement in F1 score, demonstrating both efficiency and effectiveness.

From Policy to Logic for Efficient and Interpretable Coverage Assessment

TL;DR

The paper tackles reliable, interpretable policy interpretation for medical CPT coverage by proposing a neuro-symbolic framework that couples a coverage-aware retriever with a PyKnow-based rule engine to surface governing policy language and generate auditable rationales, reducing reliance on expensive LLM inferences. It introduces a large expert-labeled retriever dataset and a contrastive ranking objective to select relevant policy passages, followed by automatic attribute extraction and rule generation that feed a symbolic inference engine. Key findings show a 44% reduction in inference cost and a 4.5% improvement in F1 when using the proposed hybrid approach, while maintaining human adjudication authority. The work demonstrates that combining targeted retrieval with symbolic reasoning yields scalable, interpretable, and cost-effective support for policy review in healthcare, addressing both reliability and efficiency concerns in real-world adjudication.

Abstract

Large Language Models (LLMs) have demonstrated strong capabilities in interpreting lengthy, complex legal and policy language. However, their reliability can be undermined by hallucinations and inconsistencies, particularly when analyzing subjective and nuanced documents. These challenges are especially critical in medical coverage policy review, where human experts must be able to rely on accurate information. In this paper, we present an approach designed to support human reviewers by making policy interpretation more efficient and interpretable. We introduce a methodology that pairs a coverage-aware retriever with symbolic rule-based reasoning to surface relevant policy language, organize it into explicit facts and rules, and generate auditable rationales. This hybrid system minimizes the number of LLM inferences required which reduces overall model cost. Notably, our approach achieves a 44% reduction in inference cost alongside a 4.5% improvement in F1 score, demonstrating both efficiency and effectiveness.
Paper Structure (16 sections, 7 figures, 2 tables)

This paper contains 16 sections, 7 figures, 2 tables.

Table of Contents

  1. Introduction
  2. Related Work
  3. Translating Natural Language into First-Order Logic
  4. LLMs for Legal Reasoning
  5. LLMs for Rule-Making
  6. Coverage Documents
  7. Methodology
  8. Policy Text Retrieval Phase
  9. Symbolic Phase
  10. Experimental Settings
  11. Results
  12. Overall Performance
  13. Where do the rules fail?
  14. Conclusion
  15. Prompts
  16. ...and 1 more sections

Figures (7)

  • Figure 1: High level pipeline of CPT code analysis in healthcare.
  • Figure 2: A sample text from Diabetes Services subsection of a plan document.
  • Figure 3: Pipeline of our neuro-symbolic approach. This system supports human reviewers by making the underlying policy logic interpretable.
  • Figure 4: The prompt used for attribute generation.
  • Figure 5: An example of a rule generated. Each rule is in this similar format.
  • ...and 2 more figures