Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Enriching Patent Claim Generation with European Patent Dataset

Lekang Jiang, Chengzu Li, Stephan Goetz

TL;DR

This paper tackles the need for cross-jurisdiction evaluation of patent claim generation beyond USPTO data. It introduces EPD, a large-scale dataset of European patents granted by the EPO in 2024 with English text and rich metadata, designed for multiple patent tasks including claim generation. Through extensive experiments comparing models trained on HUPD-DCG and EPD, the authors show that fine-tuning on EPD yields significant improvements in claim quality and cross-domain generalization, sometimes outperforming GPT-4o on the EPD test set. The study also introduces a difficult subset to simulate real-world drafting challenges and discusses implications for model robustness and future research in European patent claim generation.

Abstract

Drafting patent claims is time-intensive, costly, and requires professional skill. Therefore, researchers have investigated large language models (LLMs) to assist inventors in writing claims. However, existing work has largely relied on datasets from the United States Patent and Trademark Office (USPTO). To enlarge research scope regarding various jurisdictions, drafting conventions, and legal standards, we introduce EPD, a European patent dataset. EPD presents rich textual data and structured metadata to support multiple patent-related tasks, including claim generation. This dataset enriches the field in three critical aspects: (1) Jurisdictional diversity: Patents from different offices vary in legal and drafting conventions. EPD fills a critical gap by providing a benchmark for European patents to enable more comprehensive evaluation. (2) Quality improvement: EPD offers high-quality granted patents with finalized and legally approved texts, whereas others consist of patent applications that are unexamined or provisional. Experiments show that LLMs fine-tuned on EPD significantly outperform those trained on previous datasets and even GPT-4o in claim quality and cross-domain generalization. (3) Real-world simulation: We propose a difficult subset of EPD to better reflect real-world challenges of claim generation. Results reveal that all tested LLMs perform substantially worse on these challenging samples, which highlights the need for future research.

Enriching Patent Claim Generation with European Patent Dataset

TL;DR

This paper tackles the need for cross-jurisdiction evaluation of patent claim generation beyond USPTO data. It introduces EPD, a large-scale dataset of European patents granted by the EPO in 2024 with English text and rich metadata, designed for multiple patent tasks including claim generation. Through extensive experiments comparing models trained on HUPD-DCG and EPD, the authors show that fine-tuning on EPD yields significant improvements in claim quality and cross-domain generalization, sometimes outperforming GPT-4o on the EPD test set. The study also introduces a difficult subset to simulate real-world drafting challenges and discusses implications for model robustness and future research in European patent claim generation.

Abstract

Drafting patent claims is time-intensive, costly, and requires professional skill. Therefore, researchers have investigated large language models (LLMs) to assist inventors in writing claims. However, existing work has largely relied on datasets from the United States Patent and Trademark Office (USPTO). To enlarge research scope regarding various jurisdictions, drafting conventions, and legal standards, we introduce EPD, a European patent dataset. EPD presents rich textual data and structured metadata to support multiple patent-related tasks, including claim generation. This dataset enriches the field in three critical aspects: (1) Jurisdictional diversity: Patents from different offices vary in legal and drafting conventions. EPD fills a critical gap by providing a benchmark for European patents to enable more comprehensive evaluation. (2) Quality improvement: EPD offers high-quality granted patents with finalized and legally approved texts, whereas others consist of patent applications that are unexamined or provisional. Experiments show that LLMs fine-tuned on EPD significantly outperform those trained on previous datasets and even GPT-4o in claim quality and cross-domain generalization. (3) Real-world simulation: We propose a difficult subset of EPD to better reflect real-world challenges of claim generation. Results reveal that all tested LLMs perform substantially worse on these challenging samples, which highlights the need for future research.
Paper Structure (32 sections, 4 figures, 13 tables)

This paper contains 32 sections, 4 figures, 13 tables.

Table of Contents

  1. Introduction
  2. Related Work
  3. Patent Claim Generation
  4. Patent Datasets
  5. Dataset
  6. Construction
  7. Statistics
  8. Experiments
  9. Datasets for Claim Generation
  10. Models
  11. Evaluation Metrics
  12. Results
  13. Performance on European and US Patent Datasets
  14. Performance on Difficult and Easy Samples of EPD
  15. Qualitative Analysis
  16. ...and 17 more sections

Figures (4)

  • Figure 1: LLM-as-a-judge evaluation results on EPD and HUPD-DCG.
  • Figure 2: LLM-as-a-judge performance difference between EPD and HUPD-DCG (EPD minus HUPD-DCG)
  • Figure 3: LLM-as-a-judge performance on difficult and easy subsets of EPD.
  • Figure 4: LLM-as-a-judge performance difference between difficult and easy subsets (difficult minus easy).