AI Can Learn Scientific Taste
Jingqi Tong, Mingzhe Li, Hangcheng Li, Yongzhuo Yang, Yurong Mou, Weijie Ma, Hongji Chen, Xiaoran Liu, Qinyuan Cheng, Ming Zhang, Qiguang Chen, Weifeng Ge, Qipeng Guo, Tianlei Ying, Tianxiang Sun, Yining Zheng, Zhiheng Xi, Xinchi Chen, Jun Zhao, Ning Ding, Xuanjing Huang, Yu-Gang Jiang, Xipeng Qiu
cs.CL
Submitted: 2026-08-19
Updated: 2026-08-20
Comments: 47 pages, 5 figures
Code: https://github.com/tongjingqi/AI-Can-Learn-Scientific-Taste
License: http://creativecommons.org/licenses/by/4.0/
The gist: Scientific discovery depends on expert judgement and foresight, which we call scientific taste: the ability to judge and propose research ideas with the potential for long-term scientific impact.
Terminology
Abstract
Scientific discovery depends on expert judgement and foresight, which we call scientific taste: the ability to judge and propose research ideas with the potential for long-term scientific impact. Scientific taste is largely concentrated among highly experienced researchers, whose expertise is usually limited to a few specialised fields. If AI could learn scientific taste, it could reduce reliance on human experts and accelerate scientific discovery. Whether AI can learn this ability remains an open question. We introduce Reinforcement Learning from Community Feedback (RLCF) to learn judgement and ideation. Scientific Judge learns from community feedback, such as citations. Scientific Thinker learns to propose research ideas with high potential impact. Experiments show that Scientific Judge outperforms strong LLM baselines and that learned judgement generalises to future-year papers, other community metrics, and unseen fields. Furthermore, Scientific Thinker proposes research ideas with higher potential impact than those proposed by baselines. These results suggest that AI can learn scientific taste, marking an important step towards AI systems that could help accelerate scientific discovery.
Sources
- Search-R1: Training LLMs to Reason and Leverage Search Engines with Reinforcement Learning
- WisPaper: Your AI Scholar Search Engine
- Agent Laboratory: Using LLM Agents as Research Assistants
- The AI Scientist-v2: Workshop-Level Automated Scientific Discovery via Agentic Tree Search
- Can LLMs Generate Novel Research Ideas? A Large-Scale Human Study with 100+ NLP Researchers
- The Ideation-Execution Gap: Execution Outcomes of LLM-Generated versus Human Research Ideas
- WorldPM: Scaling Human Preference Modeling
- Learning to summarize from human feedback
- RewardBench: Evaluating Reward Models for Language Modeling
- RMB: Comprehensively Benchmarking Reward Models in LLM Alignment
- Reward Reasoning Model
- Generative Reward Models
- Generative Verifiers: Reward Modeling as Next-Token Prediction
- Inference-Time Scaling for Generalist Reward Modeling
- RM-R1: Reward Modeling as Reasoning
- Unified Reward Model for Multimodal Understanding and Generation
- Unified Multimodal Chain-of-Thought Reward Model through Reinforcement Fine-Tuning
- DeepSeekMath: Pushing the Limits of Mathematical Reasoning in Open Language Models
- Pref-GRPO: Pairwise Preference Reward-based GRPO for Stable Text-to-Image Reinforcement Learning
- The Invisible Leash: Why RLVR May or May Not Escape Its Origin
Related papers
- Exploring Solution Divergence and Its Effect on Large Language Model Problem Solving
- Ishigaki-IDS-Bench: A Benchmark for Generating Information Delivery Specification from BIM Information Requirements
- Subliminal Steering: Stronger Encoding of Hidden Signals
- MedStruct-S: A Benchmark for Key Discovery, Key-Conditioned QA and Semi-Structured Extraction from OCR Clinical Reports
- The End of Transformers? On Challenging Attention and the Rise of Sub-Quadratic Architectures
- Untangling the Mechanisms of Misleading Context in Medical Question Answering