On the Role of Citations in Preference Data

arXiv:2608.21376 · cs.CL, cs.AI · Submitted 2026-07-06 · Read on arXiv

Listen

Transcript

Introduction to the show: ident: AI Radio. Generated commentary on the latest Artificial Intelligence papers.

Tom: Hi, I'm Tom.

Jane: And I'm Jane, here with Lu, Meng and our in-house large language model, Lalam.

Tom: We're looking at the paper: "On the Role of Citations in Preference Data".

Jane: Here's the gist of the paper. The gist The role of citations in preference data for human judges and LLMs within scientific question answering reveals that humans prefer more diverse but fewer overall citations, while LLMs show some stronger citation-related preferences depending on the data and specific models.

Tom: General Setup The study investigates how citation-related features influence human preferences in scientific question answering by drawing on mixed-effects models (MEMs) to compare judgments across two datasets: SciArena and ResearchQA The analysis requires datasets that satisfy three specific conditions: 1.

Jane: Responses must contain citations 2.

Tom: Each query must have multiple responses so that pairwise comparisons are possible 3.

Title and authors: Tom: Let's talk about who wrote this. It’s Yu Hou, Hal Daumé III, Rachel Rudinger, and William Walden from the University of Maryland and Johns Hopkins University.

Jane: It's a team that seems to have a solid background in NLP and machine learning research, which is exactly what you need when you're looking at how models process complex information like citations.

Tom: Yeah, they’re researchers who are using mixed effects models to interrogate these questions about citation features influencing pairwise judgments on scientific question answering responses.

Lu: They set up experiments comparing human preferences against judges from four different open-weight LLMs: Llama3 point 3-70B, Gemma3-27B, Qwen3-30B, and Skywork-Critic-70B.

Meng: Comparing them to humans is smart because it shows where the AI's internal decision process starts to diverge from what a person would naturally prefer.

Lalam: The key finding here is that LLMs show some stronger citation preferences compared to humans, but those preferences aren't universal; they depend on the specific data and the model being tested.

The paper's summary: Tom: So, what’s the core of what this paper says? It investigates how citation-related features affect preference data when judging scientific answers.

Jane: Simply put, they found two main things: first, human judges prefer responses that have a wider variety of cited sources but don't use as many citations in total.

Tom: That’s right. And second, the LLM experiments showed that these citation features influence their preferences differently than they do for humans, which suggests potential biases in how we train reward models for these systems.

Lu: They used specific predictors like citation diversity, which is measured by the Shannon diversity index—basically how varied the sources are—and citation count.

Meng: And they also looked at things specific to ResearchQA, like citation mismatch, which measures when an in-text citation doesn't match what’s in the reference list.

Lalam: The paper highlights that for ResearchQA human judges prefer more diverse citations but fewer total citations, and the citation mismatch feature significantly correlates with a response's probability of being chosen.

The paper's improvements: Tom: The authors suggest a few ways to improve how we use this preference data. They think we should design better instructions for annotators so they understand cited content more easily before and during the collection process.

Jane: That makes sense because if the annotators don't clearly know what’s being cited, their judgments might be based on assumptions instead of actual content quality.

Lu: For LLM judges specifically, they suggest incorporating external tools, like giving them access to cited documents through external tools, to help them better approximate human evaluations for long-form responses.

Meng: Having the AI look up the source directly should help bridge that gap where the AI only sees text without knowing what’s actually behind it.

Lalam: They also suggest incorporating a bias check based on response content uniqueness for LLM judges because they found that this predictor shows much stronger correlations across all LLM judges than those of human judges.

Conclusion: Tom: So, to wrap up the paper "On the Role of Citations in Preference Data," humans prefer more diverse citations but fewer overall citations, and LLMs process this information in a fundamentally different way.

Jane: That suggests that while we can use this data to train reward models, we need to be careful about how we interpret those preferences because the mechanics are different for the AI.

Lu: The paper points toward using these findings to inform reward model training by generating responses that satisfy both diversity and conciseness preferences at the same time.

Meng: I’m thinking if we want reliable AI outputs, maybe we need to focus on making sure our judging systems are sensitive to those citation details, like what they suggested about content uniqueness.

Lalam: This work really contributes to understanding the role citations play in preference data from both human and LLM judges, showing that even when comparing outputs without the original sources, these citation features matter a lot.

Yu Hou, Hal Daumé III, Rachel Rudinger, William Walden

University of Maryland · Johns Hopkins University

cs.CL, cs.AI

Submitted: 2026-07-06

Updated: 2026-10-21

Comments: EMNLP 2026

Code: https://github.com/yale-nlp/SciArena

License: http://creativecommons.org/licenses/by/4.0/

Importance score: 90/100

The gist: The gist The role of citations in preference data for human judges and LLMs within scientific question answering reveals that humans prefer more diverse but fewer overall citations, while LLMs show

Key concepts

Citation Diversity (CD): This measures how varied the sources cited in a response are. The study found that human judges prefer responses with a higher degree of citation diversity, meaning they use citations from a wider range of sources rather than relying on just a few specific ones.
Citation Count (CC): This refers to the total number of citations present in a response. Human judges indicated they prefer responses that contain fewer total citations compared to those with more, suggesting they value conciseness alongside diversity.
Citation Mismatch (CM): This predictor specifically looks for discrepancies between what is cited inline and what appears in the reference list. High mismatch negatively correlates with a response being chosen, indicating that consistency between in-text citations and the bibliography is important for preference.
Mixed-Effects Models (MEMs): These are statistical tools used to compare judgments across different datasets like SciArena and ResearchQA while accounting for varying factors. They help researchers determine how citation features influence preferences while controlling for other variables.

Summary

The gist The role of citations in preference data for human judges and LLMs within scientific question answering reveals that humans prefer more diverse but fewer overall citations, while LLMs show some stronger citation-related preferences depending on the data and specific models.

General Setup

The study investigates how citation-related features influence human preferences in scientific question answering by drawing on mixed-effects models (MEMs) to compare judgments across two datasets: SciArena and ResearchQA

The analysis requires datasets that satisfy three specific conditions:

Responses must contain citations
Each query must have multiple responses so that pairwise comparisons are possible
Because we wish to compare LLM judge behavior with that of humans, each response pair must have a clear human preference vote

LLM Inference Experiment

Preferences were elicited from four open-weight LLMs: Llama3.3-70B (Grattafiori et al., 2024), Gemma3-27B (Team et al., 2025), Qwen3-30B (Yang et al., 2025), and Skywork-Critic-70B (Shiwen et al., 2024)

To control for biases arising from presentation order, experiments were run for both possible orderings of responses in the prompt, yielding 12, 249 × 2 = 24, 498 and 169 × 2 = 338 judgments per LLM for SciArena and ResearchQA

Fixed-Effects Predictors

For SciArena, the fixed-effects predictors considered are citation diversity (CD), citation count (CC), citation age (CA) [SciArena Only], response length (RL, in characters)—a common source of bias in preference data (Dubois et al., 2024); and response content (RC) measures the difference in unique content between two responses

For ResearchQA only, the fixed-effects predictors include citation mismatch (CM) [ResearchQA only], which captures discrepancies between inline citations and the reference list in ResearchQA responses

LLM Judge Preferences

Compared to humans, LLM judges show similar but stronger preferences for diverse citations

For non-citation-related variables in SciArena, Llama, Gemma, and Qwen all exhibit strong preferences for longer responses

Case Study: ResearchQA

For ResearchQA human judges prefer more diverse citations but fewer total citations

The citation mismatch predictor significantly negatively correlates with a response’s probability of being chosen

Discussion

The study concludes that human judges prefer more diverse citations but fewer total citations

LLM judges process this information in a fundamentally different manner

The findings suggest that both human and LLM judges may benefit from more detailed instructions for assessing citation quality before and during the preference collection process

Limitations

The study focuses exclusively on SciArena and ResearchQA to study human and LLM preferences

The results reflect correlations, not causal relationships

In conclusion, this work contributes to our understanding of the role of citations in preference data from human and LLM judges

Improvements for AI systems

Diverse citation preference modeling for human judges can inform reward model training by indicating that humans prefer more diverse citations but fewer citations overall (Page 1). This allows reward models to generate responses that satisfy both diversity and conciseness preferences simultaneously, improving the quality of post-training LLM outputs.
LLM judging systems can be tuned to align with human judgment by incorporating external tools, such as access to cited documents through external tools (Page 5). This enables LLMs to better approximate human evaluations when assessing citation quality in long-form responses.
Preference data collection protocols for both humans and LLMs should include clear citation-related rubrics or tools that assist annotators in more easily understanding cited content (Page 5). This reduces reliance on implicit, potentially biased assessments by domain experts during the collection phase.
For LLM judges, incorporating a bias check based on response content uniqueness can improve reliability; for instance, the response content predictor shows much stronger correlations than those of human judges across all LLM judges (Page 3). This helps in selecting the most reliable judge model based on its sensitivity to semantic differences.
LLM judge selection should move beyond simple accuracy metrics by using MEM analysis to evaluate biases of interest and in facilitating more nuanced and principled LLM judge selection (Page 7). This ensures that the chosen judging model is appropriate for the specific downstream task requirements.

Abstract

Many NLP tasks require systems to provide attribution in their outputs--i.e. citations to grounding sources. Attribution serves as a bulwark against model hallucination and as a means for users to verify the credibility of model outputs. Yet, it is unclear how humans and LLMs evaluate citations when comparing outputs, a process central to reward modeling and modern LLM post-training. This paper studies the role of citations in the preferences of human judges and four open-source LLMs within the context of scientific question answering, leveraging mixed effects models to investigate the influence of citations on pairwise judgments. Among our key findings are (1) that humans prefer more diverse citations but fewer overall, and (2) that LLMs show some citation-related preferences compared to humans, despite lacking access to the sources, but these preferences depend on the data and specific models. We further discuss the implications of our findings for preference data collection.