Enriching Patent Claim Generation with European Patent Dataset

arXiv:2505.12568 · cs.CL · Submitted 2025-05-18 · Read on arXiv

Listen

Transcript

Introduction to the show: ident: AI Radio. Generated commentary on the latest Artificial Intelligence papers.

Tom: We're looking at the paper: "Enriching Patent Claim Generation with European Patent Dataset".

Jane: Here's the gist of the paper. The gist The EPD dataset enriches patent claim generation by providing high-quality, jurisdictionally diverse European patent data, which shows that LLMs fine-tuned on it significantly outperform those trained on previous datasets and even GPT-4o in claim quality and cross-domain generalization.

Tom: Drafting patent claims is a time-intensive, costly, and skill-dependent process that requires professional expertise The complexity of drafting and revising patent claims makes the process both time-consuming and costly Recent advances in natural language processing (NLP) and large language models (LLMs) have shown promise in general text generation and patent-related tasks Early approaches primarily focused on fine-tuning neural models to generate claims from abstracts However, Jiang et al.

Jane: (2025b) found that abstract-based claim generation often leads to suboptimal results due to the generic nature of abstracts Consequently, recent work and the increase of context length in LLMs have enabled description-based claim generation, which can leverage the detailed technical content in patent descriptions to enhance claim completeness and quality.

Tom: Advantages of the EPD Dataset EPD presents rich textual data and structured metadata to support multiple patent-related tasks, filling a critical gap by providing a benchmark for European patents EPD enriches the field in three critical aspects: (1) Jurisdictional diversity, as patents from different offices vary in legal and drafting conventions, (2) Quality improvement, as EPD offers high-quality granted patents with finalized and legally approved texts, and (3) Real-world simulation, where a difficult subset is proposed to better reflect real-world challenges of claim generation.

Title and authors: Tom: So we're looking at this paper, "Enriching Patent Claim Generation with European Patent Dataset". It sounds like they are trying to fix the problem of patent claim generation which is really time-consuming and requires professional skill.

Jane: Right, it’s about taking something that’s mostly from the United States and adding something new to make things better for everyone. So, this paper introduces a European patent dataset called EPD to help train large language models on these claims.

Lu: What's interesting is how they frame this whole thing as filling a gap because existing work has largely relied on datasets from the USPTO, and they say EPD provides a benchmark for European patents.

Meng: A benchmark sounds good, but what does that actually mean for us in terms of what we’re building right now? Is this just another dataset to throw at the current models?

Lalam: Well, according to the paper's summary on page two they found that EPD enriches the field in three critical aspects: jurisdictional diversity, quality improvement, and real-world simulation <ref:2505.12568#pg1,enriches the field in three critical aspects>.

Tom: Let’s unpack those three things a bit then. They are focusing on how patents from different offices have different legal and drafting conventions, which is jurisdictional diversity.

Jane: And they’re also focusing on the quality of the data itself, because they emphasize that EPD offers high-quality granted patents with finalized and legally approved texts, which is a big deal for accuracy.

Lu: They also mention real-world simulation by proposing a difficult subset to better reflect the real-world challenges of claim generation, which suggests they aren't just using clean data.

Meng: So they’re trying to make the training data more robust against messy, actual patent language rather than just perfect examples.

Lalam: And when we look at the specific construction on page three they get their data by retrieving English-based European patents granted between January two thousand twenty-four and August two thousand twenty-four through the EPO’s Open Patent Services or OPS API <ref:2505.12568#pg3,English-based European patents granted between January 2024 and August 2024>.

Tom: That sounds like a very structured way to gather it, pulling in both the application and granted versions of patents if related applications aren't provided.

Jane: And they processed all that retrieved data into a structured JSON format with fields like publication numbers, title, classification codes, and claims for easy reading by the AI.

Lu: And then when comparing it to other datasets mentioned on page two like HUPD-DCG or Patent-CR, they show some specific differences in how many claims are per patent—eleven point five versus thirteen point six—but the average length per claim is actually higher in EPD at eighty-four point five tokens compared to eighty-one point one tokens from HUPD-DCG.

Title and authors: Meng: That difference in token length is significant, because it affects how much context the model has to work with during generation or revision tasks.

Tom: And structurally, they also observed that EPD exhibits deeper syntactic trees, at forty-seven point zero compared to thirty-eight point two from HUPD-DCG, though they also noted lower structural complexity and better readability in their findings on page two.

Jane: So the implication there is that while the structure might be more complex for the model to parse initially, it leads to better results when generating claims.

Lu: The main finding they highlight is that LLMs fine-tuned specifically on EPD significantly outperform those trained on HUPD-DCG and even GPT-4o across both test sets <ref:2505.12568#pg2>.

Meng: That’s a strong claim, but we need to see the numbers behind that performance boost, not just the qualitative statement of outperforming previous models.

Tom: The paper gives us those specific metrics on page two: Llama-three point one-8B-FT fine-tuned on EPD achieved a BLEU score of forty-seven point two two, an R-one score of sixty-seven point nine four, an R-L score of fifty-one point five two, and a BERTScore of ninety point four zero on the EPD test set itself.

Jane: That’s quite a high performance on those specific metrics when tested directly against the EPD data they used for fine-tuning, which suggests the quality of that training really paid off.

Lu: What I find particularly interesting is the cross-domain generalization ability they observed: their EPD-trained model actually shows performance degradation when tested on HUPD-DCG, suggesting higher quality training data might enable stronger generalization across patent jurisdictions.

Meng: So it suggests that the specific quality of the training set matters a lot, and that a model trained on one jurisdiction might not transfer perfectly to another without further tuning.

Tom: They also introduced a difficult subset of EPD to mimic real-world challenges, where patents are classified as difficult if no sentence meets an ROUGE-L score above zero point six with the first claim, and they found that all tested LLMs performed substantially worse on those challenging samples.

Jane: That shows that even with high quality data, there are still complex real-world drafting challenges that current models struggle to handle when the prompt is tough enough.

Lu: In their qualitative analysis, they noted improvements in feature completeness and structural coherence for the fine-tuned models, giving an example where Llama-three point one-8B omitted key technical details like that the cartridge can be recharged and manually activate in a gold claim.

Meng: That kind of omission is exactly what we worry about when we deploy models; they might miss critical dependencies or specific functional language that matters for real use.

Tom: So, to wrap up on this paper, the main implication is that using high-quality granted claims from European Patent Office datasets like EPD can create LLMs that are significantly better at generating patent claims compared to those trained on older or less diverse data.

Title and authors: Jane: They show that focusing the training on these specific, high-quality texts helps models learn more stable linguistic and logical patterns, leading to better claim generation performance in both the specific domain and when they are tested across different legal contexts.

Lu: The paper also points out a limitation: DPO-based Reinforcement Learning from Human Feedback does not necessarily contribute to meaningful improvements in claim generation when applied directly to these fine-tuned models, which suggests that we might need more specific steps to construct an effective preference dataset for this task.

Meng: And one major caveat they mention is that the time gap and the evolution of patent language between USPTO and EPO standards can lead to lower performance for older models trained on previous data.

Tom: So, while EPD is a great foundation, we still have hurdles with real-world complexity and how different legal traditions affect model performance across jurisdictions.

Jane: And it opens up the idea that using these diverse datasets is essential if we want models capable of handling patent drafting across different global standards effectively.

Lu: Overall, this paper suggests that polished granted claims allow models to learn more stable patterns, which is a step forward for claim generation systems.

Meng: We need to keep pushing on how these models handle those difficult, real-world samples they created, because that’s where the gap between lab performance and actual deployment is usually biggest.

Lalam: So to summarize "Enriching Patent Claim Generation with European Patent Dataset", it provides a high-quality dataset from the EPO to show that fine-tuning LLMs on these texts results in measurable improvements in claim quality, even showing cross-domain generalization effects, though real world challenges remain tough for them.

Tom: Exactly. So, EPD is a key piece of the puzzle if we want better AI assistance for patent attorneys or inventors dealing with European filings.

Jane: It definitely gives us a clearer picture of what kind of data is needed to move beyond generic claim generation and toward something more specialized and accurate.

Lu: The next step seems to be investigating how we can best leverage this jurisdictional diversity without it simply causing model confusion when it encounters novel patent language styles.

Meng: From an operational standpoint, we need to figure out how much effort it takes to curate a dataset like EPD versus what the immediate performance gain is when fine-tuning on it.

Lalam: So that’s the picture from this paper on Enriching Patent Claim Generation with European Patent Dataset.

Tom: Right, moving on now we have a lot of specific data points to think about for how we can actually build better claim generation tools.

The paper's summary: Jane: So, this paper is about taking European patent data from the EPO and using it to make large language models much better at writing patent claims than they were before.

Tom: Yeah, they show that when you fine-tune a model on this specific EPD dataset, it actually outperforms GPT-4o and even other models trained on US data for claim quality metrics like BLEU score and R-one score <ref:2505.12568#pg2>.

Lu: What's really interesting is the way the authors frame EPD as solving a problem that abstract text isn't good at. They argue that description-based generation, using these detailed patent descriptions, leads to much better claims.

Meng: I mean, it’s not just about getting a higher score on a test set; it’s about whether this model actually performs when you try to deploy it in the real world where things get messy.

Lalam: They address that by introducing this subset of data called difficult samples, designed to mimic the real challenges of drafting claims, and they show that even with high-quality training, these models still struggle significantly with those hard examples.

Tom: And the qualitative analysis on those difficult samples showed that the fine-tuned models actually improved things like feature completeness and structural coherence in their generated claims.

Jane: It suggests that the polished granted claims teach the model more stable linguistic and logical patterns, which makes it better at generating claims across different settings.

Lu: They also found something about cross-domain generalization; they showed that a model fine-tuned on EPD actually performed worse when tested on US patent data, indicating that high quality training data can create a specific kind of bias in the model's knowledge.

Meng: That’s a bit of a red flag for me if I’m thinking about production deployment; it means you have to be really careful about which jurisdiction you train your AI on and how that affects its performance elsewhere.

Tom: And they also looked at using Direct Preference Optimization, or DPO, for reinforcement learning from human feedback, and the finding there is that DPO doesn't actually give meaningful improvements in claim generation when applied directly to these models.

Jane: So even when you try to use advanced alignment techniques like RLHF or DPO on top of a model trained on this data, it doesn't automatically make the claims better in this specific task.

Lu: The authors point out that the time gap and the evolution of patent language between European and US standards can cause older models trained on previous data to underperform compared to these newer ones.

Meng: It sounds like EPD isn't just a dataset; it’s a quality filter, but we still have to figure out how to handle those jurisdictional differences when building something that needs to work globally.

Tom: So the big picture here is that if you want claim generation AI that actually moves beyond generic text and gets into the technical specifics of patents, focusing on high-quality granted documents like EPD is a major way forward.

Jane: It’s important because it shows us exactly what kind of data structure—the detailed, finalized legal text—is what the models need to learn to draft patent claims accurately.

Lu: But we still have those hard samples, and figuring out how to make models robust enough for those real-world complexities is the next big research challenge.

The paper's improvements: Tom: So, we’re talking about what this paper suggests we should actually *do* with this EPD data for claim generation models.

Jane: Essentially, they say the main improvement comes from using these high-quality granted claims to teach the AI stable linguistic patterns that it can rely on.

Lu: They argue that when a model learns from finalized, legally approved texts instead of just raw abstracts, it gains better structural understanding for generating those claims.

Meng: I mean, so instead of just feeding it millions of general patent descriptions, you give it these polished examples so the output is actually usable in a legal context?

Tom: Exactly. They show that this focused training leads to measurable gains in feature completeness and technical precision in what the AI outputs.

Jane: But they also pointed out something about generalization; even though it works great on European data, we still see performance drops when it’s tested against US patent data, which highlights a real hurdle.

Lalam: The paper implies that for claim generation to be truly useful across different legal systems, we need more than just high quality in one jurisdiction; we need a way for the model to adapt its patterns.

Tom: And they also looked at reinforcement learning methods like DPO, and they found those didn't provide the expected boost in this specific task, which is a pretty important piece of data for us engineers.

Lu: It suggests that while aligning models with preferences is useful generally, for something as structured as patent claims, the underlying training data quality has a much bigger impact than just the preference tuning.

Meng: So what does this mean for us on the engineering side? It means we need to prioritize curating these high-fidelity datasets over just chasing more training samples from any source.

Jane: And they suggest that future work should focus on how to bridge those jurisdictional gaps, maybe by showing models how to handle those differences in legal drafting conventions.

Tom: I think the implication is that we need a multi-jurisdictional approach if we want these models to be deployed broadly, not just in one specific legal environment.

Lu: They leave the door open for exploring more robust ways to teach the AI about patent language structure, which is where they feel most confidence right now.

Conclusion: Tom: So we’ve looked at how this paper, "Enriching Patent Claim Generation with European Patent Dataset," uses EPO data to boost model performance on patent claims compared to other datasets and even GPT-4o <ref:2505.12568#pg2>.

Jane: Basically, they are showing that fine-tuning on these high-quality granted patents gives the AI better results in terms of quality metrics across different domains.

Lu: The main point is that the polished nature of the granted claims helps models learn more stable linguistic patterns, which is a solid foundation for generation.

Meng: It’s interesting how they found that this training method also affects cross-jurisdictional generalization; it can make a model better at one type of patent system but weaker at another.

Lalam: I'm pretty sure the data structure they use, the JSON format with all those fields like publication numbers and claims, is exactly what gives the models that stable training signal.

Tom: Right, so we’re looking at a way to make these claim generation systems more reliable by using data that actually represents final legal documents from a specific office.

Jane: It changes things for us because it shows that moving toward domain-specific, high-quality training sets is key if we want AI tools to handle real technical tasks like drafting legal documents accurately.

Lu: And they flag that the time difference between patent language standards, like between the EPO and USPTO, can still cause performance issues if you’re using older models trained on different things.

Meng: So it means when I look at deploying a model, I need to consider not just how good the initial training is, but also how well it handles those shifts between different legal standards.

Tom: To wrap up, this paper on enriching patent claim generation with European Patent Dataset shows that high-quality granted claims provide a strong signal for models to learn precise technical and legal language.

Jane: It gives us a clear direction on what kind of data we need to prioritize if we want AI to actually perform well in specialized fields like intellectual property.

Lu: The future work they suggest is figuring out how to handle that jurisdictional diversity more smoothly, rather than just seeing performance drop when it encounters something new.

Meng: I still think we need to focus on making sure these models can handle those difficult real-world samples they created, because lab results don't always translate directly to what breaks at scale.

Lalam: I’m confident that the findings on the structural coherence and completeness improvements are solid; I expect those patterns to hold true in more complex scenarios.

Lekang Jiang, Chengzu Li, Stephan Goetz

University of Cambridge

cs.CL

Submitted: 2025-05-18

Updated: 2026-10-17

Comments: EMNLP 2025 Findings. 18 pages, 13 tables, 4 figures

Code: https://github.com/scylj1/EPD

License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/

The gist: The gist The EPD dataset enriches patent claim generation by providing high-quality, jurisdictionally diverse European patent data, which shows that LLMs fine-tuned on it significantly outperform

Key concepts

EPD Dataset: The EPD dataset is a rich collection of 73,980 European patents with structured metadata. It is valuable because it offers jurisdictional diversity and high-quality, legally approved granted patents, filling a gap in existing training data for patent claim generation tasks.
Description-Based Generation: This approach focuses on generating patent claims by leveraging the detailed technical content found within the full patent descriptions rather than relying solely on generic abstracts. This method has been shown to lead to better claim completeness and higher quality outputs because it utilizes more specific technical information.
Cross-Domain Generalization: This refers to a model's ability to perform well when applied to patent data from different legal jurisdictions, such as the USPTO and EPO. Training on diverse data helps models learn stable patterns that allow them to generalize their knowledge effectively across these distinct legal conventions.
Difficult Samples: These are a specific subset of EPD patents intentionally selected because they are challenging for current models to generate high-quality claims from, often failing specific quality metrics. Using these samples helps researchers identify the limitations of existing LLMs and highlights areas needing further research.

Terminology

Benchmark — 1× in this paper · explained in 2 episodes
Generalization — 1× in this paper · explained in 13 episodes
Large Language Models (LLMs) — 1× in this paper · explained in 8 episodes

Summary

The gist The EPD dataset enriches patent claim generation by providing high-quality, jurisdictionally diverse European patent data, which shows that LLMs fine-tuned on it significantly outperform those trained on previous datasets and even GPT-4o in claim quality and cross-domain generalization.

Introduction to the Problem

Drafting patent claims is a time-intensive, costly, and skill-dependent process that requires professional expertise The complexity of drafting and revising patent claims makes the process both time-consuming and costly Recent advances in natural language processing (NLP) and large language models (LLMs) have shown promise in general text generation and patent-related tasks Early approaches primarily focused on fine-tuning neural models to generate claims from abstracts However, Jiang et al. (2025b) found that abstract-based claim generation often leads to suboptimal results due to the generic nature of abstracts Consequently, recent work and the increase of context length in LLMs have enabled description-based claim generation, which can leverage the detailed technical content in patent descriptions to enhance claim completeness and quality.

Advantages of the EPD Dataset

EPD presents rich textual data and structured metadata to support multiple patent-related tasks, filling a critical gap by providing a benchmark for European patents EPD enriches the field in three critical aspects: (1) Jurisdictional diversity, as patents from different offices vary in legal and drafting conventions, (2) Quality improvement, as EPD offers high-quality granted patents with finalized and legally approved texts, and (3) Real-world simulation, where a difficult subset is proposed to better reflect real-world challenges of claim generation. The dataset consists of 73,980 patents in total, with 32,988 having both the application and granted versions.

Experimental Setup and Comparison

The study compares LLMs on both the HUPD-DCG dataset (from USPTO) and the EPD dataset (from EPO) The researchers allocated 1,035 samples for testing from each set to ensure a fair comparison. When comparing claim statistics between EPD and HUPD-DCG, the average number of claims per patent is slightly lower in EPD (11.5 vs. 13.6), but the average length per claim is higher (84.5 vs. 81.1 tokens). Structurally, EPD exhibits deeper syntactic trees (47.0 vs. 38.2), while lower structural complexity and better readability are observed.

Model Performance and Fine-Tuning Effects

Experiments show that LLMs fine-tuned on EPD significantly outperform those trained on HUPD-DCG and even GPT-4o across both test sets. Specifically, Llama-3.1-8B-FT (EPD) achieved the highest BLEU (47.22), R-1 (67.94), R-L (51.52), and BERTScore (90.40) on the EPD test set. Furthermore, the EPD-trained model exhibits cross-domain generalization ability, as its performance degrades when tested on HUPD-DCG, suggesting that higher-quality training data can potentially enable stronger generalization across patent jurisdictions.

Difficult Samples and Qualitative Analysis

The researchers introduce a difficult subset of EPD to mimic real-world challenges, classifying patents as difficult if no sentence meets an ROUGE-L score above 0.6 with the first claim. All tested LLMs perform substantially worse on these challenging samples, highlighting the need for future research. Qualitative analysis of generated claims shows that fine-tuned models demonstrate improvements in multiple aspects, such as feature completeness and structural coherence. For instance, Llama-3.1-8B omits key technical details present in the gold claim, such as the cartridge can be recharged and manually activate.

Conclusion and Future Directions

Overall, the findings provide valuable insights into the limitations and opportunities in automated patent claim generation. The EPD dataset serves as a foundation for future research toward stronger claim-generation systems. The paper concludes that the polished granted claims allow models to learn more stable linguistic and logical patterns, resulting in better claim generation performance in both in-domain and cross-domain settings. The study suggests that while fine-tuning on granted claims improves quality, DPO-based RLHF does not necessarily contribute to meaningful improvements in claim generation. The paper also notes that the time gap and evolution of patent language between USPTO and EPO standards can lead to lower performance for older models trained on previous data.

Limitations

The EPD dataset only includes patents published by the European Patent Office and documented in English. For claim generation, patent descriptions are constrained to fewer than 8,000 tokens to ensure a fair comparison with previous research. Additionally, the study does not perform hyperparameter tuning during fine-tuning or inference.

Ethics Statement

Llama-3 is under the META LLAMA 3 COMMUNITY LICENSE AGREEMENT, GPT-4o is under a commercial license provided by OpenAI, and the dataset is collected from EPO’s Open Patent Services (OPS). The proposed dataset is organized for multiple patent-related tasks and is compatible with the original access and use conditions. The researchers plan to use the CCBY-NC-4.0 license.

References

The references include works such as Frumkin (1947) on early history of patents, Jiang and Goetz (2025) on natural language processing in the patent domain, and various other papers cited throughout the text.

Patent Background

Patent documents are essential for protecting intellectual property (IP) and documenting inventions They provide a detailed description of new inventions and define the scope of patent rights granted to the holder As integral components of the patenting process, these documents become publicly accessible once a patent is issued. Patents are organized into hierarchical categories to facilitate efficient searching and management.

Data Structure and Format

Each patent document is saved to a structured JSON file, named by its publication number and version, such as EP1234567B1 The data fields include publication numbers, title, kind code, publication dates, primary and secondary classification codes, patent family, application reference, applicants, inventors, citations, abstract, claims and descriptions.

More Tasks

The EPD dataset supports multiple patent-related tasks. In Patent Subject Classification task definition, the model is trained on claims consistently outperforms the model trained on abstracts in both accuracy and F1 score. This indicates that the information in the claim section is more discriminative or indicative of a patent’s subject than the abstract section.

Summarization

Patent summarization aims to generate concise yet informative abstracts from patent claims or descriptions.

Improvements for AI systems

Improved claim generation quality by fine-tuning large language models on high-quality granted patents from European Patent Office datasets like EPD, as experiments show that LLMs fine-tuned on EPD significantly outperform those trained on previous datasets and even GPT-4o in claim quality.
Enhanced cross-jurisdictional generalization by training models on diverse patent data, enabling them to better handle differences in linguistic style and legal drafting conventions between jurisdictions like the USPTO and EPO.
Increased robustness against real-world drafting challenges by exposing models to a difficult subset of EPD designed to simulate real-world situations of claim drafting, which highlights the need for future research on model handling of complex dependencies.
Improved patent subject classification accuracy by training models based on claims rather than abstracts, as results show that the model trained on claims consistently outperforms the model trained on abstracts in both accuracy and F1 score for IPC labels.
Better patent summarization by fine-tuning models specifically on recent datasets like EPD, as older models struggle with the time gap and the evolution of patent language, technology, and subject matter, leading to improved ROUGE scores compared to previous HUPD T5-Small results.
Refined model preference alignment using Direct Preference Optimization (DPO) for claim generation by analyzing performance metrics showing that DPO-based RLHF does not enhance claim generation quality when applied directly to the fine-tuned models, suggesting a need for additional refinements or processing steps required to construct an effective preference dataset.
Developed specialized evaluation capabilities by using LLM-as-a-judge evaluators like DeepseekV3 with Chain-of-Thought prompting to assess claims across multiple dimensions, including feature coverage, technical precision, specificity, clarity, dependency structure, legal terminology, and scope of protection.