Towards Automated Quality Assurance of Patent Specifications: A Multi-Dimensional LLM Framework

TL;DR

This work tackles the scalability gap in patent quality assurance by introducing a three-module LLM-based framework that jointly assesses regulatory compliance, technical coherence, and figure-reference consistency, followed by an integration module that delivers actionable improvement guidance. It validates the approach on a matched-pair dataset of 160 patents (80 human, 80 AI-generated) across eight domains, demonstrating near-perfect regulatory compliance detection (balanced accuracy ≈99.74%), solid technical coherence (≈82.1%), and strong figure-text consistency (≈91.2%). The analysis reveals distinct defect patterns by patent section and IPC domain, with AI-generated patents exhibiting more structural and cross-referencing issues than human-authored ones, especially in complex multi-component domains. While promising for automated first-pass screening and remediation guidance, the study notes limitations in dataset size and the need for refined severity distinctions and multimodal capabilities in patent drafting tools. These findings highlight practical pathways to augment AI-assisted patent drafting, reduce prosecution risk, and guide future multimodal QA research in patent specifications.

Abstract

Although AI drafting tools have gained prominence in patent writing, the systematic evaluation of AI-generated patent content quality represents a significant research gap. To address this gap, We propose to evaluate patents using regulatory compliance, technical coherence, and figure-reference consistency detection modules, and then generate improvement suggestions via an integration module. The framework is validated on a comprehensive dataset comprising 80 human-authored and 80 AI-generated patents from two patent drafting tools. Evaluation is performed on 10,841 total sentences, 8,924 non-template sentences, and 554 patent figures for the three detection modules respectively, achieving balanced accuracies of 99.74%, 82.12%, and 91.2% against expert annotations. Additional analysis was conducted to examine defect distributions across patent sections, technical domains, and authoring sources. Section-based analysis indicates that figure-text consistency and technical detail precision require particular attention. Mechanical Engineering and Construction show more claim-specification inconsistencies due to complex technical documentation requirements. AI-generated patents show a significant gap compared to human-authored ones. While human-authored patents primarily contain surface-level errors like typos, AI-generated patents exhibit more structural defects in figure-text alignment and cross-references.

Figures (1)

• Figure 1: Overview of the patent quality assurance framework. Three specialized detection modules operate in parallel to identify regulatory compliance, technical coherence, and figure-reference consistency issues, followed by a sequential integration module that synthesizes findings into comprehensive improvement recommendations.