History, Adaptation, and Hidden Gain: Two Prospective Tests of Learned Predictive Control in a Synthetic Neural Population
cs.LG, cs.SY, eess.SY
Submitted: 2026-10-09
Updated: 2026-10-09
Comments: 13 pages, 4 figures. Synthetic closed-loop neural rate-control study; code, frozen protocols and results at https://github.com/lonewolf15116/NeuroLight-AI (archived: https://doi.org/10.5281/zenodo.23263620)
Code: https://github.com/lonewolf15116/NeuroLight-AI
License: http://creativecommons.org/licenses/by/4.0/
The gist: We study learned one-step predictive control of a synthetic population of 64 leaky integrate-and-fire neurons whose light sensitivity is scaled by a hidden factor s, using two prospective
Terminology
Abstract
We study learned one-step predictive control of a synthetic population of 64 leaky integrate-and-fire neurons whose light sensitivity is scaled by a hidden factor s, using two prospective confirmations on plants simulated only after the analysis was frozen. In the first (50 plants, 11 conditions), removing the 200-ms input-output history substantially worsened a parameter-count-matched neural predictor at low and moderate gain (s = 0.6, 0.8; RMSE rose by 2.47 Hz at s = 0.6, 97.5% CI 2.43-2.51). In secondary comparisons, recursively adapted ARX had the lowest mean error of the deployable controllers in 10 of 11 conditions, tuned PI with anti-windup matched or beat the network for s >= 0.7, and training for 10,000 rather than 900 epochs lowered prediction error but worsened control. Development work suggested the network's learned action sensitivity did not track the hidden gain. A second protocol (50 new plants, 9 sensitivities, 2 noise levels) confirmed three pre-specified predictions at the primary noise level. Re-estimating a single multiplicative input gain recovered 87% (95% CI 86-88%) of the benefit of full adaptive ARX. Supplying that online estimate to the network as an extra input reduced high-gain tracking error by 1.84 Hz [1.81, 1.88]. Longer training worsened the plain network by 1.09 Hz but improved the gain-input network by 1.07 Hz, a differential of 2.16 Hz [2.13, 2.20]. Under high process noise (sigma = 2.0) the effects shrank to about 0.04 Hz or less and gain-only adaptation recovered only 24% of the adaptive-ARX benefit. In this synthetic plant, at the primary noise level, one-parameter online gain estimation accounts for most of the gap between a history-based network and full adaptive ARX, and for its prediction-control mismatch.
Sources
- Neural Network Dynamics for Model-Based Deep Reinforcement Learning with Model-Free Fine-Tuning
- Objective Mismatch in Model-based Reinforcement Learning
- Nonlinear Model Predictive Control of a Conductance-Based Neuron Model via Data-Driven Forecasting
- Model Predictive Control on the Neural Manifold
- RMA: Rapid Motor Adaptation for Legged Robots
- Preparing for the Unknown: Learning a Universal Policy with Online System Identification
- The Value Equivalence Principle for Model-Based Reinforcement Learning
- Leveling the Playing Field: Carefully Comparing Classical and Learned Controllers for Quadrotor Trajectory Tracking
- Learning Nonlinear Systems In-Context: From Synthetic Data to Real-World Motor Control
Related papers
- Polynomial-Augmented Neural Networks (PANNs) with Weak Orthogonality Constraints for Enhanced Function and PDE Approximation
- AIRL-S: Unifying Reinforcement Learning and Search-Based Test-Time Scaling via Adversarial Inverse Reinforcement Learning
- Transformers as Bayesian In-Context Experimenters: Smoothness-Adaptive Efficient ATE Estimation
- Convergence issues in Relational Concept Analysis based on AOC-posets
- Beliefs Beyond Posteriors: The Predictive Cost of ELBO Binding
- Understanding Diffusion Models via Ratio-Based Function Approximation with SignReLU Networks