STAGE: Source-grounded Text-to-JSON Artifact GEneration
cs.CL
Submitted: 2026-06-18
Updated: 2026-10-13
Comments: Preprint
Code: https://github.com/boradorish/STAGE-Eval
License: http://creativecommons.org/licenses/by-sa/4.0/
The gist: From financial filings to clinical records, legacy industries rely heavily on long, unstructured documents to store high-value information.
Terminology
Abstract
From financial filings to clinical records, legacy industries rely heavily on long, unstructured documents to store high-value information. Reliably extracting this information into structured, machine-readable representations is a key prerequisite to making the contents accessible to automated systems. JSON is a natural target for such structured extraction, yet constructing reliable and scalable text-to-JSON training data remains challenging. To address this gap, we propose STAGE (Spreadsheet-grounded Text-to-JSON Artifact GEneration), a source-grounded data generation pipeline that constructs reports and JSON schema by using LLMs for scalable synthesis while validating ground-truth values against the underlying spreadsheet. Evaluations on STAGE-Eval, our source-grounded benchmark with an 851-example test set, show that STAGE produces stronger training data than existing approaches. This improves Qwen3-4B exact match from 31.37% to 74.27% and value accuracy from 45.46% to 90.69%.
Sources
- GPT-4 Technical Report
- ScrapeGraphAI-100k: Dataset for Schema-Constrained LLM Generation
- SING-SQL: A Synthetic Data Generation Framework for In-Domain Text-to-SQL Translation
- Ground-Truth Subgraphs for Better Training and Evaluation of Knowledge Graph Augmented LLMs
- ExtractBench: A Benchmark and Evaluation Methodology for Complex Structured Extraction
- JSONSchemaBench: A Rigorous Benchmark of Structured Outputs for Language Models
- The Llama 3 Herd of Models
- The Structured Output Benchmark: A Multi-Source Benchmark for Evaluating Structured Output Quality in Large Language Models
- SLOT: Structuring the Output of Large Language Models
- Efficient Guided Generation for Large Language Models
- Qwen3 Technical Report
- DeepJSONEval: Benchmarking Complex Nested JSON Data Mining for Large Language Models
Related papers
- Exploring Solution Divergence and Its Effect on Large Language Model Problem Solving
- Ishigaki-IDS-Bench: A Benchmark for Generating Information Delivery Specification from BIM Information Requirements
- Subliminal Steering: Stronger Encoding of Hidden Signals
- MedStruct-S: A Benchmark for Key Discovery, Key-Conditioned QA and Semi-Structured Extraction from OCR Clinical Reports
- The End of Transformers? On Challenging Attention and the Rise of Sub-Quadratic Architectures
- Untangling the Mechanisms of Misleading Context in Medical Question Answering