Inertially Significant Humanoid Teleoperation via Whole Body Model Predictive Control and Online Parameter Estimation

arXiv:2610.22176 · cs.RO · Submitted 2026-10-15 · Read on arXiv

Listen

Transcript

Introduction to the show: ident: Robotics Radio. Generated commentary on the latest robotics and control papers.

Rosa: Today's paper: "Inertially Significant Humanoid Teleoperation via Whole Body Model Predictive Control and Online Parameter Estimation".

Dev: The gist The effective deployment of humanoid robots in practical tasks of interest will require a capability of sensing and adapting to variations in inertial parameters,

Rosa: First, who's behind it and why it matters.

Paper summary: Dev: So to wrap up the paper "Inertially Significant Humanoid Teleoperation via Whole Body Model Predictive Control and Online Parameter Estimation", it’s about coupling the WB-MPC with an EKF for online estimation of inertial parameters, which then feeds back into the dynamics model.

Rosa: Right. The authors are showing that this integration results in more accurate feedforward torques when there's an external load, and they provide a VR interface to let the operator focus on the task without worrying about payload compensation.

Taro: The implication is that you can get intuitive teleoperation of humanoids manipulating significant loads by letting the control system dynamically balance itself based on what it estimates about its own inertia online.

Dev: It’s a lot of work to get the estimation right, though they have to be careful with their loop rates, running the estimator and controller at different frequencies, like one hundred Hz for the core control versus one kHz for state estimation.

Rosa: And one thing they flag is that while it handles significant payloads well in simulation and hardware experiments, we still need to see how it holds up when the world misbehaves in a truly messy, real-world setting over long periods.

Taro: That’s the limitation I see. They show adaptation works for task space error shrinking from three hundred eighty-nine point two millimeters down to something around six point four millimeters, but that's on the test bench, not necessarily in a wild environment where things are unpredictable.

Dev: So the paper demonstrates a robust framework for handling those dynamic variations during teleoperation by using parameter estimation to make the controller more accurate under load conditions.

Rosa: That’s what it is. The core idea is making sure the robot’s internal model of its own physics stays up-to-date with whatever external forces it’s dealing with, even if those forces are unknown initially.

Conclusion: Rosa: So we've looked at how this paper uses Whole Body Model Predictive Control and an Extended Kalman Filter to handle those tricky inertial parameters during humanoid teleoperation.

Dev: Exactly, Rosa, it’s about making the robot’s internal physics model smarter online so that when you’re carrying something heavy or using a tool, the control system compensates for that load in real time.

Taro: What I find interesting is how they handle things when the world isn't what they expect. They're not just trying to follow a path; they're estimating how much extra mass or inertia is messing up the dynamics right now and feeding that estimate back into the control loop.

Rosa: Right, so you’ve got this system where the operator can focus on the task—like reaching for something specific—and the robot handles all that dynamic balance and load compensation behind a closed loop.

Dev: The authors, they focused on integrating an EKF with Whole Body MPC specifically to get those consistent time-varying parameters, which is what makes it physically sound during those heavy movements.

Taro: And they showed some solid results in simulation and hardware testing; the task space error actually shrinks significantly when this adaptation is on compared to when it's off.

Rosa: It means that for someone just listening, this isn't just about a robot following a command; it’s about building a system that can dynamically adjust its own understanding of gravity and inertia while you move things.

Dev: That’s the core idea—giving the humanoid system enough intelligence to know what it’s actually carrying so it can move correctly without crashing or sagging.

Taro: But we have to remember, they still have those numerical robustness extensions, like estimating deviations from a nominal body and regularizing parameters when they are hard to measure well.

Rosa: True, even with all that estimation math, the real world is messy; we still need to see how this holds up when things get truly unpredictable outside of the clean lab environment.

Dev: That’s what we'll be digging into next—the practical limits of this whole setup and where it might break down in a real industrial or field situation.

James Foster, Beomyeong Park, Luigi Penco, Robert Griffin

Robotics Lab, Florida Institute for Human & Machine Cognition, USA · University of West Florida, USA

cs.RO

Submitted: 2026-10-15

Updated: 2026-10-15

License: http://creativecommons.org/licenses/by-nc-nd/4.0/

Importance score: 90/100

The gist: The gist The effective deployment of humanoid robots in practical tasks of interest will require a capability of sensing and adapting to variations in inertial parameters, as occurs during heavy tool

Key concepts

Whole Body Model Predictive Control (WB-MPC): This is a control method that repeatedly solves an optimization problem at every moment. It uses a full model of the robot's body to decide the best sequence of future actions, balancing the need to maintain dynamic balance with accurately tracking desired movements from the operator.
Extended Kalman Filter (EKF): The EKF is used for online estimation of time-varying inertial parameters, which are difficult to measure directly. It takes noisy sensor data and uses a physical model to continuously estimate these unknown properties, providing the controller with an updated understanding of the robot's actual dynamics.
Feedforward Torque Compensation: This technique involves calculating and applying specific torques in advance based on known or estimated external loads. By using the online parameter estimates from the EKF, the system can proactively compensate for unknown payloads, leading to more accurate movement execution and better stability.

Summary

The gist The effective deployment of humanoid robots in practical tasks of interest will require a capability of sensing and adapting to variations in inertial parameters, as occurs during heavy tool use or when carrying significant payload.

How it works

The framework integrates a whole body model predictive controller (WB-MPC) with an extended Kalman filter (EKF) for online estimation of fully physically consistent timevarying inertial parameters.
The WB-MPC trades off dynamic balance objectives with tight tracking of operator-specified taskspace objectives.
An extended Kalman filter (EKF) coupled in closed-loop with the WB-MPC estimates fully physically consistent inertial parameters online and passes them to the WB-MPC’s rigid body dynamics model, allowing feedforward torque compensation of a priori unknown loads.
The EKF uses a measurement model where the residual torque is approximated by the estimated contribution when substituted into (2) and enforces the use of an EKF as opposed to a regular Kalman filter.
The framework includes extensions to increase numerical robustness, such as estimating deviations from a nominal body using the generalized log-Cholesky parameterization.

Whole Body Model Predictive Control

The WB-MPC repeatedly solves a discrete-time optimal control problem (OCP) at each control instant and from the current estimated state xˆ. The dynamics model is obtained by manipulating the equations of motion (1) to obtain the forward dynamics. Because M and c depend on the inertial parameters π, it is through fdyn that the online estimates of these parameters enter the controller. The running cost lk of (8) is mostly a weighted sum of least-squares terms.

Extended Kalman Filter

The EKF adopts the physically consistent inertial EKF formulation and introduces two extensions to improve robustness under teleoperated motion. One extension involves estimating deviations from a nominal body using the generalized log-Cholesky parameterization of [24]. Another extension is regularizing weakly-observable parameters by appending a fictitious measurement that the parameter deviation is zero and augments the EKF update as ∆τ k0=h(θ est k)θ est k+νk, νk∼N (0, Rk), with the augmented measurement Jacobian and covariance.

Virtual Reality Teleoperation

The VR operator interface provides an intuitive control interface that allows an operator to command desired motion without the burden of compensating for added payload, while dynamic balance control is delegated to the WB-MPC. The VR operator’s tracked device poses are resolved into jointspace targets by an inverse kinematics (IK) routine running on a separate operator-side computer. The WB-MPC consumes only the upper-body portion of this solution.

Practical Implementation Details

The framework is implemented on two machines, with the state estimator and low-level control running on a 1 kHz thread and the EKF and WB-MPC running on a 100 Hz thread. The VR kinematic streaming toolbox runs on a separate operator desktop computer and streams its references via ROS2 messages at a best-effort rate of 50 Hz. The WB-MPC horizon length K = 30 nodes with an Euler discretization of ∆t = 20 ms which resolves to a prediction horizon of 0.6 s. The low-level controller closes the loop on the slower WB-MPC solution by interpolating cubic Hermite splines through the planned q⋆ and v⋆ setpoints and tracking them with a PD law that supplements the feedforward torque from the WB-MPC.

Results

Simulation experiments showed that without adaptation, the loaded left forearm sags significantly from its unloaded path, accumulating a vertical taskspace root mean square error (RMSE) of 389.2 mm. With adaptation enabled, the horizontal taskspace error is less severe and shrinks over 10× to 6.4 mm. Posture adjustment under load with the EKF enabled allows the controller to account for the added mass by modifying its internal posture, driving the pelvis backward in order to maintain the center of mass within the support polygon and its pelvis upright <ref:2610.

Improvements for AI systems

The system can perform more accurate feedforward torques for task demands under external load by integrating a whole body model predictive controller (WB-MPC) with an extended Kalman filter (EKF) for online estimation of fully physically consistent timevarying inertial parameters.
The improved AI system enables the operator to command desired motion without the burden of compensating for added payload, while dynamic balance control is delegated to the WB-MPC, allowing intuitive VR teleoperation of humanoids manipulating significant loads.
The system can achieve superior trajectory tracking under load, as simulation results showed that with adaptation, the loaded left forearm sags significantly from its unloaded path only accumulating a vertical taskspace root mean square error (RMSE) of 389.2 mm without adaptation, compared to a reduction to 6.4 mm with adaptation.
The improved system can maintain dynamic balance while carrying asymmetric loads, as the EKF allows the controller to account for the added mass by modifying its internal posture: driving the pelvis backward in order to maintain the center of mass within the support polygon and its pelvis upright.
The system can provide a robust method for handling unknown loads during operation, as demonstrated by successfully tracking incremental hardware loading where the estimated forearm mass tracks the ground-truth loading staircase closely up to 96% accuracy.

Abstract

The effective deployment of humanoid robots in practical tasks of interest will require a capability of sensing and adapting to variations in inertial parameters, as occurs during heavy tool use or when carrying significant payload. Such payloads constitute a significant proportion of the combined payload-robot system, the effects of which cannot be ignored by delegating to feedback control. This paper showcases a general solution to this problem by integrating a whole body model predictive controller (WB-MPC) with an extended Kalman filter (EKF) for online estimation of fully physically consistent time-varying inertial parameters. These estimates are fed through to the WB-MPC's full rigid body dynamics model, resulting in more accurate feedforward torques for task demands under external load. Additionally, a virtual reality (VR) interface is provided that enables an operator to provide task-specific references without regard to payload while delegating dynamic balance to the WB-MPC. The framework is validated with simulation and hardware experiments on a humanoid system.