ListQA: A Benchmark for Evaluating List-Formatted Factual Knowledge Retrieval in Large Language Models
cs.AI, cs.CL
Submitted: 2026-10-13
Updated: 2026-10-13
Comments: Accepted at NeurIPS 2026, Evaluations and Datasets Track. Project Page: https://listqa.github.io
Code: https://github.com/amazon-science/ListQA-Eval
Project page: https://listqa.github.io
License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
Terminology
Sources
- Measuring short-form factuality in large language models
- Towards a Holistic Evaluation of LLMs on Factual Knowledge Recall
- How often do Answers Change? Estimating Recency Requirements in Question Answering
- The FACTS Grounding Leaderboard: Benchmarking LLMs' Ability to Ground Responses to Long-Form Input
- Reasoning Factual Knowledge in Structured Data with Large Language Models
- CLR-Fact: Evaluating the Complex Logical Reasoning Capability of Large Language Models over Factual Knowledge
- Instruction-Following Evaluation for Large Language Models
- WildIFEval: Instruction Following in the Wild
- KCIF: Knowledge-Conditioned Instruction Following
- LiveBench: A Challenging, Contamination-Limited LLM Benchmark
- When Facts Change: Probing LLMs on Evolving Knowledge with evolveQA
- Auxiliary task demands mask the capabilities of smaller language models
- DeepSeek-V3 Technical Report
- DeepSeek-V3.2: Pushing the Frontier of Open Large Language Models
- Ministral 3
- Magistral
Related papers
- MAVEN-T: Reinforced Heterogeneous Distillation for Real-Time Multi-Agent Trajectory Prediction
- Model Discovery Agent: LLM-assisted Bayesian experiment design for data-efficient discovery of mechanistic world models
- Agents' Last Exam
- The Clinician's Veto: Navigating Trust, Liability, and Uncertainty in Autonomous AI Prescribing
- MindHelper: Closed-Loop Embodied Mental-State Reasoning for Precision Intervention
- Incumbent Advantage: Brand Bias and Cognitive Manipulation Dynamics in LLM Recommendation Systems