Scientific feedback reveals productive regimes beyond training conventions

arXiv:2512.15756 · cs.LG, cs.AI · Submitted 2025-12-12 · Read on arXiv

Listen

Transcript

Introduction to the show: ident: AI Radio. Generated commentary on the latest Artificial Intelligence papers.

Tom: Today's paper: "Scientific feedback reveals productive regimes beyond training conventions".

Jane: The gist The model exhibits emergent design-space expansion: despite being trained exclusively on configurations with a fixed number of gadolinium burnable absorber (Gd) rods,

Tom: First, who's behind it and why it matters.

Title and authors: Tom: Now, let's talk about the title and who wrote this. "Scientific feedback reveals productive regimes beyond training conventions". It suggests that just following standard training rules isn't enough to find good solutions in these engineering problems.

Jane: That sounds like it’s challenging the assumptions we usually make about how we train these kinds of models for science, right? What does that mean for us?

Tom: It means they are finding ways where the model can discover fundamentally new design topologies, not just variations of what it was trained on. They are moving beyond fixed configuration spaces.

Lu: They’re pointing to examples like AlphaGo and AlphaZero, where AI discovered novel strategies that went beyond human intuition in game spaces thirty-four thirty-five <ref:2512.15756#pg3>. This paper is applying that idea to nuclear core design.

Meng: So they're saying the conventional training methods are just too limited for the complexity of nuclear physics? That's a strong claim.

Lalam: They are showing that when you couple Monte Carlo data with language model training, you can get results that beat traditional deterministic and metaheuristic frameworks one <ref:2512.15756#pg1>.

The paper's summary: Jane: So to summarize what the paper is saying, ReactorFold proposes this generative framework to discover nuclear reactor cores. It reformulates the design as a sequence modeling problem for language models.

Tom: They use Monte Carlo data and then fine-tune using parameter-efficient methods. Then they use Direct Preference Optimization DPO, using OpenMC as an oracle to guide the generation process one <ref:2512.15756#pg1>.

Lu: The key result they highlight is emergent design-space expansion. The DPO-aligned model can autonomously adjust the number of gadolinium burnable absorber rods even when it was only trained on configurations with a fixed number of rods #pg1.

Meng: Wait, so it learns to change the inventory based on constraints? That's a big leap from just learning patterns.

Lalam: Yes, they show that this model learns how poison loading suppresses reactivity rather than just memorizing pattern-outcome correlations one <ref:2512.15756#pg1>. It suggests a causal understanding of the physics is developing.

The paper's improvements: Tom: The paper shows several ways this approach improves on existing methods. First, it’s about synthesizing layouts in a single forward pass instead of needing numerous evaluations.

Jane: So, if you could get a layout from one run instead of hundreds, that’s a huge speed boost for the search process. They claim a six-fold improvement in fitness over the genetic algorithm baseline with only one thousand high-fidelity simulations #pg10.

Lu: The structural basis for this is that the language model treats the core design as a global sequence generation task, which is different from traditional metaheuristics that operate on local modification bases

thirty-seven–forty-one: <ref:2512.15756#pg3>.

Meng: So instead of getting stuck in a local optimum with a genetic algorithm, it explores globally by treating the whole design as one sequence. That’s what I need to hear when deploying something at scale.

Lalam: Plus, they discover high-performing asymmetric configurations that they say deviate from conventional symmetric loading heuristics #pg10. They suggest optimal physical performance often requires deviating from those human-intuitive geometric rules thirty-eight <ref:2512.15756#pg3>.

Conclusion: Tom: So to wrap up, the paper on "Scientific feedback reveals productive regimes beyond training conventions" shows that foundation models can exhibit physics-driven agency. They’ve proven that they can acquire transferable physical intuition from simulation data one <ref:2512.15756#pg1>.

Jane: It seems the main implication is that sequence-based generative models might offer a unified framework for discrete combinatorial design problems across different engineering domains. It’s not just about searching efficiently within a space we define ourselves.

Lu: I think the capacity to question implicit assumptions is as valuable as the model's ability to search efficiently within a defined space #pg14. The authors are suggesting that this capability to question what we assume might be just as important for discovery thirty-eight <ref:2512.15756#pg3>.

Meng: From my side, it shows that models can go beyond mere pattern mimicry. We need to think about how we couple these generative models with multi-physics simulators for more comprehensive design exploration. That’s the practical path forward.

Lalam: I agree with Meng on that coupling. The current iteration of ReactorFold is a proof-of-concept, but coupling it with multi-physics simulators will likely enable much deeper design exploration fourteen. That's where the real power lies.

Department of Nuclear Engineering, Hanyang University

cs.LG, cs.AI

Submitted: 2025-12-12

Updated: 2026-10-14

Code: https://github.com/sixticket/reactor

License: http://creativecommons.org/licenses/by/4.0/

Importance score: 87/100

The gist: The gist The model exhibits emergent design-space expansion: despite being trained exclusively on configurations with a fixed number of gadolinium burnable absorber (Gd) rods, it autonomously adjusts

Key concepts

Human-imposed design conventions: These are the unwritten rules or established patterns that guide how humans typically design things, like reactor layouts. AI models often learn these conventions from their training data. The paper explores how feedback can help the AI move beyond these learned habits to find novel, potentially better solutions.
Learned search distribution: This refers to the set of possible designs or choices that an AI model learns to prioritize when searching for a solution. If the training data only shows certain types of designs, the model's learned distribution will favor those types. The study investigates how feedback can shift this distribution toward better, untapped areas.
Scientific performance feedback: This is information about how well a design performs in a real-world scientific context, like reactor physics or chemical yields. Instead of just rewarding designs that look like the training data, this feedback tells the AI which choices lead to superior results, even if those choices are unconventional.
Productive regions beyond conventions: These are areas in the design space that are not represented in the original training data but prove to be scientifically advantageous. The paper demonstrates that by using performance feedback, an AI can autonomously explore and identify these new, highly effective design solutions that human intuition might miss.

Terminology

Core — 1× in this paper · explained in 2 episodes
Direct Preference Optimization — 1× in this paper
Direct Preference Optimization (DPO) — 1× in this paper · explained in 10 episodes
Fine-Tuning — 1× in this paper · explained in 5 episodes
Parameter-Efficient Fine-Tuning — 1× in this paper · explained in 3 episodes

Summary

The gist The model exhibits emergent design-space expansion: despite being trained exclusively on configurations with a fixed number of gadolinium burnable absorber (Gd) rods, it autonomously adjusts Gd inventory to satisfy strict power-peaking constraints.

ReactorFold Framework

ReactorFold is a generative framework that reformulates fuel-assembly design as a sequence modeling problem for language models <ref:2512.15756#pg4>. It integrates Monte Carlo–based physical feedback with language-model–driven topology generation, enabling the direct synthesis of feasible reactor-core layouts without iterative search <ref:2512.15756#pg5>. The pipeline involves a curriculum-style, three-stage training process <ref:2512.15756#pg5>.

Curriculum-Based Training Pipeline

The training pipeline progresses through three stages to align the language model with reactor physics constraints <ref:2512.15756#pg5>.

First, the process involves Base Full Fine-Tuning (FFT) on a large corpus of low-fidelity layouts to acquire geometric syntax <ref:2512.15756#pg5>.
Second, Low-Rank Adaptation (LoRA) is used on high-fidelity data to refine physical correlations <ref:2512.15756#pg5>.
Finally, Alignment via Direct Preference Optimization (DPO) closes the loop with OpenMC as an external oracle in a preference-based optimization step <ref:2512.15756#pg7>.

Emergent Physical Reasoning and Constraint Relaxation

The model demonstrates emergent design dynamics by autonomously expanding the search space beyond human-defined inventory limits <ref:2512.15756#pg8>. Specifically, the DPO-aligned model learns to propose layouts with substantially more Gd pins than those present in the training corpus when required to satisfy stringent power-peaking and enthalpy-rise constraints <ref:2512.15756#pg9>. This capability suggests that the language model has internalized a causal understanding of how poison loading suppresses reactivity rather than merely memorizing pattern-outcome correlations <ref:2512.15756#pg9>.

Comparative Performance and Design Discovery

The ReactorFold model achieves superior neutronic balance compared to traditional evolutionary algorithms and randomized symmetric baselines <ref:2512.15756#pg10>. Within the same budget of 1,000 high-fidelity simulations, ReactorFold attains a six-fold improvement in fitness over the GA baseline <ref:2512.15756#pg10>. The model discovers high-performing asymmetric configurations that deviate from conventional symmetric loading heuristics <ref:2512.15756#pg10>. This suggests that optimal physical performance often requires deviating from human-intuitive geometric rules <ref:2512.15756#pg11>.

Structural Basis for Superior Performance

The superior performance is attributed to the fundamental difference in how the design space is traversed <ref:2512.15756#pg11>. Traditional metaheuristic approaches like GA operate on a local modification basis, often leading to entrapment in local optima <ref:2512.15756#pg10>. In contrast, the language model treats the core design as a global sequence generation task <ref:2512.15756#pg11>. The self-attention architecture enables the model to learn how spatially separated fuel components jointly affect global figures of merit such as keff and Fq <ref:2512.15756#pg11>. Furthermore, the introduction of Gd inventory as a learnable variable creates negative correlations between the inventory and key physical objectives, effectively decoupling the constraints <ref:2512.15756#pg9>.

Conclusion and Future Directions

ReactorFold demonstrates that foundation models can transcend mere pattern mimicry to exhibit physics-driven agency <ref:2512.15756#pg4>. The framework suggests that sequence-based generative models may offer a unified framework for discrete combinatorial design problems across engineering domains <ref:2512.15756#pg11>. Future work will aim to address complexities by leveraging larger foundation models and multimodal tokenizers capable of processing high-dimensional reactor states <ref:2512.15756#pg14>. The study concludes that this capability to question implicit assumptions may prove as valuable as the model’s ability to search efficiently within a defined space <ref:2512.15756#pg14>. The current iteration serves as a foundational proof-of-concept with specific limitations regarding three-dimensional full-core design and coupled thermal-hydraulic feedback <ref:2512.15756#pg14>. The authors anticipate that coupling generative models with multi-physics simulators will enable more comprehensive design exploration <ref:2512.15756#pg14>. The superior performance of ReactorFold can be attributed to the fundamental difference in how the design space is traversed <ref:2512.15756#pg11>. The model's emergent behaviour provides empirical evidence that foundation models can acquire transferable physical intuition from simulation data <ref:2512.15756#pg9>. The authors suggest that sequence-based generative models may offer a unified framework for discrete combinatorial design problems across engineering domains <ref:2512.15756#pg14>. This capacity to question implicit assumptions may prove as valuable as the model’s ability to search efficiently within a defined space <ref:2512.15756#pg14>. The authors anticipate that coupling generative models with multi-physics simulators will enable more comprehensive design exploration <ref:2512.15756#pg14>. The superior performance of ReactorFold can be attributed to the fundamental difference in how the design space is traversed <ref:2512.15756#pg10>. The model's emergent behaviour provides empirical evidence that foundation models can acquire transferable physical intuition from simulation data <ref:2512.15756#pg9>. The authors suggest that sequence-based generative models may offer a unified framework for discrete combinatorial design problems across engineering domains <ref:2512.15756#pg14>. This capacity to question implicit assumptions may prove as valuable as the model’s ability to search efficiently within a defined space <ref:2512.15756#pg14>. The authors anticipate that coupling generative models with multi-physics simulators will enable more comprehensive design exploration <ref:2512.15756#pg14>. The superior performance of ReactorFold can be attributed to the fundamental difference in how the design space is traversed <ref:2512.15756#pg10>. The model's emergent behaviour provides empirical evidence that foundation models can acquire transferable physical intuition from simulation data <ref:2512.15756#pg9>. The authors suggest that sequence-based generative models may offer a unified framework for discrete combinatorial design problems across engineering domains <ref:2512.15756#pg14>. This capacity to question implicit assumptions may prove as valuable as the model’s ability to search efficiently within a defined space <ref:2512.15756#pg14>. The authors anticipate that coupling generative models with multi-physics simulators will enable more comprehensive design exploration <ref:2512.15756#pg14>. The superior performance of ReactorFold can be attributed to the fundamental difference in how the design space is traversed <ref:2512.15756#pg10>. The model's emergent behaviour provides empirical evidence that foundation models can acquire transferable physical intuition from simulation data <ref:2512.15756#pg9>. The authors suggest that sequence-based generative models may offer a unified framework for discrete combinatorial design problems across engineering domains <ref:2512.15756#pg14>. This capacity to question implicit assumptions may prove as valuable as the model’s ability to search efficiently within a defined space <ref:2512.15756#pg14>. The authors anticipate that coupling generative models with multi-physics simulators will enable more comprehensive design exploration <ref:2512.15756#pg14>. The superior performance of ReactorFold can be attributed to the fundamental difference in how the design space is traversed <ref:2512.15756#pg10>. The model's emergent behaviour provides empirical evidence that foundation models can acquire transferable physical intuition from simulation data <ref:2512.15756#pg9>. The authors suggest that sequence-based generative models may offer a unified framework for discrete combinatorial design problems across engineering domains <ref:2512.15756#pg14>. This capacity to question implicit assumptions may prove as valuable as the model’s ability to search efficiently within a defined space <ref:2512.15756#pg14>.

Improvements for AI systems

The improved system can perform generative inverse design by reformulating fuel-assembly design as a sequence modeling problem for language models, specifically by using a curriculum-style, three-stage training pipeline that couples Monte Carlo reactor physics with language model training.
The system exhibits emergent design-space expansion, as the DPO-aligned model can autonomously adjust Gd inventory to satisfy strict power-peaking constraints despite being trained exclusively on configurations with a fixed number of rods.
The system can discover high-performing asymmetric configurations that challenge conventional symmetric loading heuristics, demonstrating the ability to transcend human-imposed design constraints.
The model can achieve superior solution quality by synthesizing layouts in a single forward pass, as it contrasts with traditional methods requiring numerous evaluations, achieving a six-fold improvement in fitness over the GA baseline within the same evaluation budget.

Abstract

Human-imposed design conventions can restrict the alternatives represented in scientific training data and shape the search preferences learned by generative models. We show that scientific performance feedback can redirect the learned search distribution toward productive regions beyond those conventions, without explicitly rewarding departures from them. We test this process in reactor lattice design with simulated feedback and chemical reaction-condition selection with measured yields, using a pretrained sequence policy and a small non-language policy, respectively. In both settings, adaptation improves scientific performance while increasing generation of choices excluded from the corresponding supervised data. In measured reactions, probability on task-specific ligand choices excluded from training rises from 0.29 to 0.70-0.71 as expected yield error decreases. Controls in the lattice setting show that the productive region identified during adaptation provides reusable information to random search, while aligned generation produces useful candidates more frequently than informed random sampling. Removing the target penalty redirects four of five matched runs, linking the direction of adaptation to the scientific objective. Together, these results show how scientific feedback reshapes inherited search preferences and reveals productive alternatives beyond training conventions, with attribution controls distinguishing learned redistribution from candidate availability and reusable search information.