Edit Policy-Aligned Text Simplification

arXiv:2512.06228 · cs.CL · Submitted 2025-12-06 · Read on arXiv

Listen

Transcript

Introduction to the show: ident: AI Radio. Generated commentary on the latest Artificial Intelligence papers.

Tom: We're looking at the paper: "Edit Policy-Aligned Text Simplification".

Jane: Here's the gist of the paper. The gist The proposed framework introduces a method leveraging Large Language Model-as-a-Judge to automatically construct policy-aligned training data, enabling the creation of simplification systems that adapt to diverse policies without requiring costly human annotation or parallel corpora.

Tom: How it works The core of the approach is a three-step framework designed for policy-aligned sentence simplification, focusing on two distinct edit policies: lexical-paraphrasing and overall-rewriting.

Jane: This method aims to learn a model that can generate ideal simplifications under any given policy by using LLMs as judges to create preference data.

Title and authors: Tom: So, we're looking at this paper today, "Edit Policy-Aligned Text Simplification," and it’s about how to make AI systems simplify text in a way that actually fits what you need.

Jane: Right. Essentially, the authors are tackling this challenge where you want a system to simplify a sentence, but you have different goals for simplification—like just swapping out hard words versus completely rewriting the whole thing—and getting that control is really hard right now.

Lu: The core idea they propose is using a Large Language Model as a Judge to automatically create training data that matches those different policies without needing any human annotators or huge parallel texts.

Meng: Without human annotation, that sounds like it could be a massive time and money saver for building these kinds of simplification tools.

Lalam: The paper sets up this three-step framework where you first gather a pool of candidate simplifications from several different Large Language Models, then an AI judge picks the best one based on specific rules, and finally, you use that preference data to fine-tune your target model.

Tom: So, the first thing we need to get is this candidate pool. It collects simplifications for every sentence under every policy you want to test.

Jane: And then the AI judge comes in, and it doesn't just guess; it evaluates those candidates based on principles—for example, rewarding lexical changes if they replace a complex word, or structural changes if they fix a long sentence structure.

Lu: The paper uses these guidelines to pick a preferred candidate and a dispreferred one for every source sentence under every policy you are testing.

Meng: That preference data is what the authors use to train their main model using something called Adaptive Rejection Preference Optimization, or ARPO, which they call a CPO variant that incorporates SimPO loss.

Tom: It sounds like they’re using this AI feedback loop to align the model with those specific simplification goals, moving away from just training on standard text.

Jane: And the results are pretty interesting because they show that even smaller open-source models, like Phi-three-mini with three point eight billion parameters, can perform better than much larger proprietary models on certain tasks.

Lu: Specifically, the paper shows that this approach helps small models surpass GPT-4o on lexical-oriented simplification while still keeping performance comparable on overall rewriting tasks <ref:2512.06228#pg1,surpass GPT-4o on lexical-oriented simplification while>.

Meng: That’s good news for deployment because it means these kinds of useful simplification tools could run on less powerful hardware than what we usually expect.

Tom: The paper also shows improvements when they compare their models against vanilla versions, for instance, Phi-three-mini got an SARI score improvement of eight point zero on lexical-oriented simplification and two point seven on overall rewriting tasks compared to the baseline models.

Jane: That points to how much better this preference optimization method is at steering the model in the right direction than just using standard training methods alone.

Title and authors: Lu: They’ve also used these LLM-as-a-Judge metrics to check alignment across three dimensions: lexical, structural, and overall simplification goals, which gives a detailed look at how well they meet those specific policies.

Meng: It’s a good way to measure quality when you don't have human graders because you’re judging against pre-defined principles.

Tom: They also found that using reasoning-based preference data, specifically data from PO Think, performed better than non-reasoning modes, like PO No-think.

Jane: That suggests that for these complex evaluations, having a judge with some kind of reasoning ability is actually helpful for getting a better outcome.

Lu: And they’ve shown that this method generalizes well to sentences outside the original test domains, like those from the SimPA and Newsela datasets, meaning the policies learned transfer effectively to new kinds of text.

Meng: That generalization is key because if a system can adapt its simplification style across different topics, it becomes much more versatile for real-world use.

Tom: But they also pointed out some limitations in their work, specifically that reasoning judges aren't perfect and can sometimes misjudge the actual word-level difficulty of a word.

Jane: And there’s another point they made about prompt engineering strategies, like using few-shot prompting or iterative feedback, which they found less robust than this preference optimization method for overall rewriting.

Lu: They also mentioned that while sentence simplification is well suited for this framework, extending it to document-level simplification might require changing the LLM-as-a-Judge guidelines to account for things like discourse structure and global coherence.

Meng: So, the paper shows a strong path forward for building flexible text simplifiers that can handle different rules, but we have to be careful about what they didn't get right yet.

Tom: To wrap up, this paper "Edit Policy-Aligned Text Simplification" shows how to build policy-driven training data automatically using an LLM as a judge to create simplification systems that adapt to different rules without needing human annotation or massive parallel corpora.

Jane: It’s about making sure the simplification system actually follows the specific instructions you give it, whether that’s just lexical changes or a full rewrite.

Lu: The authors prove that this works for different model sizes and show concrete gains on performance metrics like SARI scores when compared to baseline models.

Meng: For deployment, it shows that small models can be very effective on specific simplification tasks, which is something we need to consider when choosing our tools.

Lalam: This paper moves us toward creating systems where the policy isn't just a setting you choose; it's something the AI learns to follow directly from preference data.

Tom: So, that’s what we have today on "Edit Policy-Aligned Text Simplification." Next up, we’re looking at some work on embedding models for race prediction.

Jane: We'll be diving into how those text embeddings are being used there...

The paper's summary: Tom: So, to recap, this paper is about using an LLM as a judge to automatically generate training data for simplifying text according to different rules without needing any human writers or massive datasets.

Jane: Right, it’s about making simplification systems that actually follow specific instructions—like just changing words or completely rewriting the sentence—by teaching them from AI feedback instead of just standard training methods.

Tom: What they do is this three-step process: first, they collect a pool of candidate simplifications from various LLMs for every sentence and every policy you want to test.

Jane: Then, an AI judge looks at all those candidates and picks the best one based on specific rules—like rewarding lexical changes or structural changes—to create preference data.

Tom: And finally, they use that preference data to fine-tune their target model using a method called Adaptive Rejection Preference Optimization, which is a CPO variant.

Jane: What this means for us is that we can build these simplification tools on smaller models, like Phi-three-mini, and they can actually beat bigger proprietary models on certain tasks.

Tom: It shows the robustness of the approach across different model families and sizes, which is interesting because it suggests a path to deploying effective simplification tools on less resource-intensive hardware.

Jane: And they found that using reasoning judges, where the AI has to think through word difficulty or structure, actually leads to better results than non-reasoning modes.

Tom: That means for these complex evaluations, having a judge with some kind of reasoning ability is beneficial because it helps generate higher quality preference data.

Jane: They also showed that this method works well when the text is totally new, like sentences from the SimPA or Newsela datasets, meaning the rules we teach are transferable to different kinds of text.

Tom: But they did flag some things too. Reasoning judges aren't perfect and can sometimes misjudge word-level difficulty, which is a limitation they have to deal with.

Jane: And there’s also the point about prompt engineering strategies, like using few-shot prompting or iterative feedback, which the authors found less robust than this preference optimization method for overall rewriting tasks.

Tom: So, it's not just about getting a simplified sentence; it's about aligning the system with a specific policy in a way that’s scalable and doesn't rely on expensive human annotation.

Jane: This framework also opens the door to simplifying entire documents down to sentences, if we just adjust those judging guidelines to account for things like global coherence and discourse structure.

Tom: It’s a big step toward making text simplification systems that are truly adaptable, instead of just brittle tools that only work for one specific type of simplification.

Jane: And because this works on different model sizes, it’s not just a theoretical exercise; it's practical for building tools that can run on the hardware we actually have access to.

Tom: We need to keep an eye on how these preference optimization techniques handle the nuances of different edit policies as they try to generalize across more complex text structures.

The paper's improvements: Tom: So, to recap, they aren't just stopping at getting better scores; they are proposing ways to actually improve how these simplification systems perform across different tasks and text types.

Jane: Right, it’s not just about making one model work better; it’s about building a system that handles more complex needs by adjusting the training process itself.

Tom: They're suggesting that this preference optimization method is much more stable than other things like iterative feedback or simple prompt engineering when you're doing overall rewriting.

Jane: That means for production, if you want a reliable system for rewriting text, you should probably stick with this preference-based approach because it seems to be the more solid foundation.

Tom: The paper also shows that they can extend this idea from just simplifying single sentences to document-level simplification by modifying those judging guidelines.

Jane: That’s huge because it means we could eventually use this same AI judge logic to handle whole paragraphs, focusing on things like how ideas flow together or what the main theme of a whole chapter is.

Tom: What they are suggesting is that if you change the rules for the judge to look at document structure instead of just sentence structure, you can make these systems much more capable.

Jane: And it's not just about text simplification; they’re hinting that this same mechanism could apply to other controllable text generation tasks too.

Tom: They are arguing that we shouldn't be locked into just sentence-level work if we want powerful, adaptable AI tools in general.

Jane: So, the implication is that the next step for this research isn't just proving it works on sentences; it’s figuring out how to make the rules flexible enough to handle bigger pieces of writing.

Tom: It moves us from a specific application—simplifying a sentence—to a more general method for creating policy-aligned models across different text scales and complexity levels.

Conclusion: Tom: So we’re wrapping up on "Edit Policy-Aligned Text Simplification," which is basically this framework that uses an LLM judge to automatically create training data for text simplification under different rules, without needing human writers.

Jane: Right, it’s about taking those specific policies you have—like only focusing on word choice or only rewriting the whole sentence—and making the AI learn exactly how to do that through these AI feedback loops.

Tom: The main implication is that we can build these simplification systems that adapt to diverse policies and work well on smaller, open-source models, which is a big deal for deployment.

Jane: That means we don't have to wait for massive proprietary models to get the best results in every scenario; we can use more accessible hardware effectively.

Tom: And they showed that using reasoning judges provides better data than just non-reasoning ones, which points toward using AI with some kind of thinking ability being actually helpful for these evaluations.

Jane: That makes sense; if the judge can reason about why a sentence is complex or how to best simplify it, the preference data will be much higher quality.

Tom: Meng, you mentioned deployment earlier. If this framework is stable and we can generate the data automatically, that lowers the cost per fine-tuning run significantly.

Meng: I agree; if I can generate policy-aligned training without paying for hundreds of annotators, that makes it viable for real-world scaling because it cuts down on the annotation overhead.

Lu: From a representation learning standpoint, this suggests that we can explicitly teach the model not just what to say, but *how* to adjust its output based on a desired structural or lexical goal.

Jane: So, when we look at the future work they mentioned about document-level simplification, it seems like the next natural step is just adapting those judging guidelines for broader context.

Tom: And I'm curious what happens if we try to apply this to something totally different, maybe image editing or three dee placement?

Jane: That’s a good question, Tom; it shows the versatility of the underlying AI judge mechanism across different modalities.

Meng: But I’d need to know how much compute that takes; if it needs massive resources just for document-level coherence, then deploying it on my startup's current infrastructure is a non-starter right now.

Lalam: I’m confident in the direction; the preference data generation method seems very sure of its own answer regarding alignment, especially since it works across different model sizes.

Tom: So we've looked at how they built this framework for text simplification, and it looks like a solid way to get policy-driven training data without human input.

Jane: It definitely shows us that the future isn't just about making bigger models; it’s about teaching them smarter ways to follow instructions through structured feedback.

Tom: Next up, we’re moving away from text and looking at how embedding models are actually improving our ability to predict race outcomes in data.

Xuanxin Wu, Yuki Arase, Masaaki Nagata

University of Osaka · Institute of Science Tokyo · NTT Inc.

cs.CL

Submitted: 2025-12-06

Updated: 2026-10-21

Comments: Accepted by AACL-IJCNLP 2026

Code: https://github.com/fe1ixxu/CPO_SIMPO3https:

License: http://creativecommons.org/licenses/by/4.0/

The gist: The gist The proposed framework introduces a method leveraging Large Language Model-as-a-Judge to automatically construct policy-aligned training data, enabling the creation of simplification systems

Key concepts

LLM-as-a-Judge: A reasoning LLM is used to evaluate candidate simplifications based on specific principles. It acts as an automated grader, selecting the preferred simplification over others according to predefined guidelines, which generates preference data for training.
Policy Alignment: This refers to training a model to follow specific simplification rules, such as changing complex words (lexical) or restructuring sentences (overall rewriting). The framework uses LLM judges to generate data that aligns the target model with these diverse policy requirements.
Preference Optimization: This is the process of fine-tuning a target model using preference data generated by the LLM judges. It employs advanced techniques like ARPO, which stabilizes training and ensures the model learns to generate simplifications that meet the desired policy criteria.
Candidate Pool C(xi): This is a collection of different simplification options generated by various large language models for a single source sentence under a specific policy. This pool provides the variety needed for the LLM judge to select preferred and dispreferred examples.

Terminology

LLM-as-a-Judge — 1× in this paper · explained in 4 episodes
Robustness — 1× in this paper · explained in 6 episodes

Summary

The gist The proposed framework introduces a method leveraging Large Language Model-as-a-Judge to automatically construct policy-aligned training data, enabling the creation of simplification systems that adapt to diverse policies without requiring costly human annotation or parallel corpora.

How it works

The core of the approach is a three-step framework designed for policy-aligned sentence simplification, focusing on two distinct edit policies: lexical-paraphrasing and overall-rewriting. This method aims to learn a model that can generate ideal simplifications under any given policy by using LLMs as judges to create preference data.

Candidate Pool for Preference Data: The process begins by collecting a set of candidate simplifications from multiple large language models (LLMs), varying in size and family, for every source sentence and every policy. This results in a candidate pool defined as C(xi) = (2) where yi,k ∼ Mk(xi, sp), k = 1,..., K.
LLM-as-a-Judge: A reasoning-capable LLM is employed as the judge to evaluate the candidates based on carefully designed principles that specify edit types and their associated rewards or penalties. For example, lexical principles reward replacing complex words with simpler alternatives, while structural principles reward simplifying difficult structures or splitting long sentences. The judge selects a preferred candidate y(i)w and a dispreferred candidate y(i)l from the pool C(xi) according to guidelines G: (3).
Preference Optimization: The resulting preference dataset D = (x(i), y(i)w, y(i)l)(N i=1 for each policy p is used to fine-tune the target model πθ. This optimization utilizes Adaptive Rejection Preference Optimization (ARPO), a CPO variant that integrates SimPO loss, to align the model with the generated preference data.

Policy Alignment and Model Performance

This method significantly enhances policy alignment capabilities of small-scale LLMs, allowing them to surpass large proprietary models like GPT-4o on lexical-paraphrasing while achieving comparable performance on overall-rewriting tasks. The framework is robust across different model families and sizes, as verified by both automatic metrics and human evaluations. For instance, the approach showed that SARI scores improved by +8.0 for Phi3-3.8B on lexical-paraphrasing and +2.7 for Phi3-3.8B on overall-rewriting compared to vanilla models.

Evaluation and Comparison

The framework is evaluated using standard benchmark datasets like the Turk test set for lexical-paraphrasing, assessed with SARI, and the ASSET test set for overall-rewriting, assessed with LENS. The results demonstrate that the LLM-as-a-Judge approach consistently outperforms human-written parallel corpora because it offers scalability and better quality control. Furthermore, training on reasoning-based preference data (PO Think) consistently outperforms non-reasoning modes (PO No-think), suggesting that complex evaluations benefit from reasoning judges.

Generalization and Limitations

The method demonstrates strong generalization to out-of-domain sentences, such as those from the SimPA and Newsela datasets, showing that learned policies can transfer effectively to different domains. However, limitations exist; reasoning judges are not flawless, sometimes misjudging word-level difficulty. Moreover, prompt engineering strategies like few-shot prompting or iterative feedback are less robust than the preference optimization method, as iterative feedback can degrade performance on overall rewriting for every model tested.

Future Directions

The study suggests that while sentence-level simplification is well-suited for this framework, extending it to document-level simplification is possible by adapting the LLM-as-a-Judge guidelines to account for global coherence and discourse structure. Future research should also explore applying this framework to other languages and other controllable text generation tasks.

Under review, POLICY-BASED SENTENCE SIMPLIFICATION: REPLACING PARALLEL CORPORA WITH LLM-AS-A-JUDGE, xuanxin.wu@ist.osaka-u.ac.jp arase@c.titech.ac.jp, masaaki.nagata@ntt.com, 6 Dec 2025>

Under review, Table 1: Simplifications by our model under two edit policies (Phi-3-mini-3.8B (Abdin et al., 2024a)). We highlight the main simplification edits in each part of the sentence using different colors. Red: Deletions Green: Paraphrasing Blue: Split Source Shade sets the main plot of the novel in motion when he impetuously defies that law, and inadvertently initiates a chain of events that leads to the destruction of his colony’s home, forcing their premature migration, and his separation from them., 1>

Under review, Table 1: Simplifications by our model under two edit policies (Phi-3-mini-3.8B (Abdin et al.

Improvements for AI systems

Policy-driven training data generation without human annotation or parallel corpora allows simplification systems to adapt to diverse policies like lexical-paraphrasing and overall-rewriting. This enables building simplification systems that adapt to diverse simplification policies.
Small-scale open-source LLMs can surpass larger proprietary models on specific tasks, such as Phi3-mini with lexical-oriented simplification, demonstrating the robustness of our approach. This suggests a path toward deploying effective simplification tools on less resource-intensive hardware.
The framework uses reasoning-capable LLMs as judges to automatically generate high-quality preference data under each policy, replacing costly human annotation with reinforcement learning from AI feedback (RLAIF). This makes creating policy-aligned training data scalable and efficient for various models.
The method optimizes model parameters using Contrastive Preference Optimization (CPO) combined with Simple Preference Optimization (SimPO) within the Adaptive Rejection Preference Optimization (ARPO) framework, which is a CPO variant designed to mitigate its tendency to overly penalize dispreferred responses that are only marginally worse than preferred ones. This leads to more stable and high-performing fine-tuned models.
The system can be evaluated using LLM-as-a-Judge metrics that assess alignment across three dimensions: lexical, structural, and overall. This allows for a nuanced understanding of how well simplifications meet specific policy requirements, as shown by the Figure 7: Guidelines for sentence-level simplifications.
The framework can be extended to document-level simplification by modifying the LLM-as-a-Judge guidelines to include aspects like discourse relations, coreference resolution, and global coherence. This allows the system to handle more complex text structures while maintaining the same three-step pipeline.

Abstract

Text simplification aims to modify documents and sentences to make them easier to read and understand while preserving their meanings. Different applications require distinct simplification policies, e.g., lexical-oriented simplification and overall rewriting while trading off detail for simplicity. However, achieving such policy-driven control remains an open challenge. In this work, we introduce a simple yet powerful approach that leverages Large Language Model-as-a-Judge (LLM-Judge) to automatically construct policy-aligned training data, removing the need for costly human annotation or parallel corpora. Our method enables the construction of simplification models that adapt to diverse simplification policies at both the sentence and document levels. Remarkably, small-scale open-source LLMs match or surpass GPT-5, as verified by both automatic metrics and edit analysis. The consistent improvements across model families and sizes demonstrate the robustness of our approach.