Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Can AI Examine Novelty of Patents?: Novelty Evaluation Based on the Correspondence between Patent Claim and Prior Art

Hayato Ikoma, Teruko Mitamura

TL;DR

This work tackles the problem of automating patent novelty assessment by modeling the examiner-like comparison between patent claims and cited prior art using an LLM evaluation framework. It introduces a dataset derived from real USPTO examination results and evaluates both encoder-based classifiers and generative models under various prompting regimes, including zero-shot, few-shot, and supervised fine-tuning. The study finds that classification-only approaches struggle to capture claim-cited-text correspondences, while large generative models can predict novelty with reasonable accuracy and provide interpretable explanations, albeit with limitations. The findings suggest AI-assisted patent analysis can reduce examiner and applicant workloads, while underscoring the need for richer datasets, improved prompts, and larger-context models to reach robust, scalable performance.

Abstract

Assessing the novelty of patent claims is a critical yet challenging task traditionally performed by patent examiners. While advancements in NLP have enabled progress in various patent-related tasks, novelty assessment remains unexplored. This paper introduces a novel challenge by evaluating the ability of large language models (LLMs) to assess patent novelty by comparing claims with cited prior art documents, following the process similar to that of patent examiners done. We present the first dataset specifically designed for novelty evaluation, derived from real patent examination cases, and analyze the capabilities of LLMs to address this task. Our study reveals that while classification models struggle to effectively assess novelty, generative models make predictions with a reasonable level of accuracy, and their explanations are accurate enough to understand the relationship between the target patent and prior art. These findings demonstrate the potential of LLMs to assist in patent evaluation, reducing the workload for both examiners and applicants. Our contributions highlight the limitations of current models and provide a foundation for improving AI-driven patent analysis through advanced models and refined datasets.

Can AI Examine Novelty of Patents?: Novelty Evaluation Based on the Correspondence between Patent Claim and Prior Art

TL;DR

This work tackles the problem of automating patent novelty assessment by modeling the examiner-like comparison between patent claims and cited prior art using an LLM evaluation framework. It introduces a dataset derived from real USPTO examination results and evaluates both encoder-based classifiers and generative models under various prompting regimes, including zero-shot, few-shot, and supervised fine-tuning. The study finds that classification-only approaches struggle to capture claim-cited-text correspondences, while large generative models can predict novelty with reasonable accuracy and provide interpretable explanations, albeit with limitations. The findings suggest AI-assisted patent analysis can reduce examiner and applicant workloads, while underscoring the need for richer datasets, improved prompts, and larger-context models to reach robust, scalable performance.

Abstract

Assessing the novelty of patent claims is a critical yet challenging task traditionally performed by patent examiners. While advancements in NLP have enabled progress in various patent-related tasks, novelty assessment remains unexplored. This paper introduces a novel challenge by evaluating the ability of large language models (LLMs) to assess patent novelty by comparing claims with cited prior art documents, following the process similar to that of patent examiners done. We present the first dataset specifically designed for novelty evaluation, derived from real patent examination cases, and analyze the capabilities of LLMs to address this task. Our study reveals that while classification models struggle to effectively assess novelty, generative models make predictions with a reasonable level of accuracy, and their explanations are accurate enough to understand the relationship between the target patent and prior art. These findings demonstrate the potential of LLMs to assist in patent evaluation, reducing the workload for both examiners and applicants. Our contributions highlight the limitations of current models and provide a foundation for improving AI-driven patent analysis through advanced models and refined datasets.

Paper Structure

This paper contains 18 sections, 4 figures, 7 tables.

Table of Contents

  1. Introduction
  2. Task
  3. Datasets
  4. Data Source
  5. Pre-processing
  6. Methodology
  7. Models
  8. Prompting
  9. Result
  10. Human Testing
  11. Longformer, Llama2, 3(with the classification head)
  12. Llama2,3, GPT-4o
  13. Qualitative Analysis
  14. Related Work
  15. Conclusion and Future Work
  16. ...and 3 more sections

Figures (4)

  • Figure 1: An Overview of Patent Examination Process: Examiners compare patent claims with prior art and issue a non-final rejection if grounds for rejection are found. Applicants can amend the claims, and the examiner compare them with prior art again. This process repeats until a decision is made to approve or reject the patent.
  • Figure 2: Entire patent examination process and patent documents from which input data is extracted: the Input Claim is came from Original Claim(label "Non-Novel") or Amended Claim(label: "Novel"). the Input Cited Texts is came from cited prior art documents. "14/617301" and "US 2011/0079639 A1" are example document numbers; Application Number and Publication Number.
  • Figure A1:
  • Figure A3: Explanation created from the Non-Final Rejection in the prompt for the "Non-Novel" label. The additional parts of the amended claim are highlighted in gray.