When Citations Mislead? A Claim-Level Benchmark for Legal Hallucination Detection
cs.CL, cs.AI
Submitted: 2026-10-07
Updated: 2026-10-07
Comments: 9 pages, 2 figures, 7 tables. Published in the 21st International Conference on Artificial Intelligence and Law (ICAIL 2026), Singapore. Dataset: https://github.com/mmikaildemir/PARCEL
Journal ref: In Proceedings of the 21st International Conference on Artificial Intelligence and Law (ICAIL 2026), June 08-12, 2026, Singapore. ACM, New York, NY, USA
Code: https://github.com/mmikaildemir/PARCEL
Project page: https://indigobook.github.io/versions/indigobook-2.0-rev2023-2.html
License: http://creativecommons.org/licenses/by/4.0/
The gist: Large language models are increasingly used in legal research and drafting, but they can still produce claims that sound convincing without being supported by the cited source.
Terminology
Abstract
Large language models are increasingly used in legal research and drafting, but they can still produce claims that sound convincing without being supported by the cited source. We introduce PARCEL, a benchmark for checking whether a legal claim is supported by the underlying authority. Using recent New York State Court of Appeals decisions, we build a dataset of 3,396 parenthetical-style claims labeled as Supported, Refuted, or Not Found. We cast this task as a three-way natural language inference problem and evaluate several state-of-the-art LLMs in a zero-shot setting. Although the strongest models reach up to 0.97 accuracy, the results also show an important weakness: models still incorrectly mark unsupported claims as supported, even when the full opinion text is provided. Across models, missing support is harder to detect than direct contradiction, and fabricated but plausible citations cause the largest drop in performance. Overall, PARCEL provides a practical benchmark for testing claim-level groundedness in legal RAG systems.
Sources
- Bye-bye, Bluebook? Automating Legal Procedure with Large Language Models
- Hallucination-Free? Assessing the Reliability of Leading AI Legal Research Tools
- Towards Effective Extraction and Evaluation of Factual Claims
- Adaptive Chameleon or Stubborn Sloth: Revealing the Behavior of Large Language Models in Knowledge Conflicts
- Enhancing Health Fact-Checking with LLM-Generated Synthetic Data
Related papers
- Exploring Solution Divergence and Its Effect on Large Language Model Problem Solving
- Ishigaki-IDS-Bench: A Benchmark for Generating Information Delivery Specification from BIM Information Requirements
- Subliminal Steering: Stronger Encoding of Hidden Signals
- MedStruct-S: A Benchmark for Key Discovery, Key-Conditioned QA and Semi-Structured Extraction from OCR Clinical Reports
- The End of Transformers? On Challenging Attention and the Rise of Sub-Quadratic Architectures
- Untangling the Mechanisms of Misleading Context in Medical Question Answering