Compliant GMT: Making General Motion Tracking Compliant via Residual Reference Adaptation

The gist: Compliant GMT (C-GMT) is a modular framework that unifies general motion tracking with adjustable whole-body compliance through residual reference adaptation, achieving both competitive motion tracking accuracy and compliance adherence.

How it works

The core idea of C-GMT is to decompose compliant general motion tracking into two components: (i) a force-robust general motion tracker that faithfully executes reference motions under external loads, and (ii) a residual adapter that modifies the reference to produce a desired compliant response. This decomposition allows the desired compliant behavior to be specified independently of the motion-tracking objective

The system utilizes a privileged teacher to generate compliant references for the downstream GMT, which is then distilled into a residual adapter conditioned on proprioception The training process involves three stages: "first train for 500 updates using teacher-driven rollouts. Over the following 1000 updates, we linearly transition from teacher-driven to student-driven rollouts. Training then continues using student-driven trajectories until convergence"

Force-Robust General Motion Tracker

The force-robust general motion tracker is built upon SONIC [3], which encodes reference motions into a learned latent representation and quantizes them into universal motion tokens To make this tracker robust to sustained loads, the decoder is finetuned to be robust to forces while keeping its motion encoders frozen During finetuning, each hand is subjected to a combination of carried payloads and sustained external forces with a combined load constrained by a force budget that gradually increases from 10 to 30 N during training The force-robust tracker improves local tracking accuracy under nominal conditions, with only a 4.8 cm increase in global tracking error compared to the original SONIC policy

Compliant Teacher and Feasibility Tuning

A privileged teacher is constructed using a spring law and an inverse-kinematics (IK) solver to design and verify feasible compliant responses The IK objective prioritizes the desired pose of the forced link, followed by foot-placement and center-of-mass objectives for maintaining balance A critical benefit is that the desired response can be evaluated for feasibility and style before training the residual adapter This teacher is subjected to an automated simulation sweep over 4096 representative reference motions, eight force directions, and multiple stiffness values

Residual Adapter Architecture

The residual adapter learns to reproduce the reference modifications generated by the privileged teacher using proprioceptive observations and previous actions It receives a 10-step history of proprioceptive observations and previous actions, together with the motion token z and the commanded stiffness k The architecture is implemented as a lightweight MLP with residual, gating, and auxiliary force-prediction heads The modified token is calculated as z˜ = z + g ⊙ ∆z, where gates are learned through imitation objectives and are encouraged to be zero when no external force is present

Training and Results

The training objective combines action and motion-token imitation, auxiliary force prediction, residual regularization, and temporal smoothness To address oscillatory behavior, the paper introduces two complementary losses: an unnormalized anti-oscillation loss (Losc) and a normalized anti-oscillation loss (Lnorm osc) Feasibility-based force sampling was shown to reduce falls during training by approximately 13× and accelerate compliance learning, reaching over 95% give-way adherence within 6k updates C-GMT maintains tracking performance close to SONIC, with only a modest increase in global position error Furthermore, C-GMT demonstrates superior safety during dynamic motions by producing the lowest interaction forces when placed next to a rigid wall compared to SONIC and TWIST2 Finally, C-GMT shows that coordinated whole-body compliance can improve robustness to sustained external forces by allowing the robot to redistribute the disturbance through the rest of the body

The framework successfully achieves both general motion tracking and whole-body compliant responses, demonstrating improved safety and robustness in simulation and real-world experiments The research concludes that compliance at the low-level motion-tracking layer can improve the robustness of higher-level policies without sacrificing nominal task performance.

--- Page 1 ---

Compliant GMT: Making General Motion Tracking Compliant via Residual Reference Adaptation Venki Parsuram Michelle Wang Jin-Hyuk Hong Pulkit Agrawal Fig. 1: Compliant-GMT combines unified whole-body compliant responses with whole-body general motion tracking capabilities. The compliance enabled by C-GMT improves safety and robustness in manipulation, human-robot interaction, disturbance attenuation, and more. At the same time, C-GMT is a general motion tracking policy capable of performing diverse, dynamic motions. Abstract— We propose Compliant GMT (C-GMT), a modular framework for full-body compliant responses in general motion tracking. As humanoid robots move into real world deployment, they not only need to perform any motion but also respond safely and robustly to interaction forces. Existing general motion tracking policies can imitate diverse motions but are trained to be stiff, leading to unexpected and unsafe behaviors under disturbance. Prior work in compliant humanoid control trades off compliance and generality. There exist examples of full-body compliant responses, but these are heavily curated to single-reference policies. On the other hand, general motion trackers (GMT) are trained to rigidly imitate massive motion capture datasets. Our key insight is to leverage the generality of pretrained GMTs for motion tracking while introducing compliance from a residual adapter that conditions on a history of proprioception. This enables our approach, C-GMT, to achieve both general motion tracking and wholebody compliant responses. Compared to state-of-the-art general motion trackers and single-motion compliant policies, C-GMT demonstrates competitive performance in both motion tracking accuracy and compliance adherence. We further demonstrate improved safety, robustness, and error tolerance in simulation and real-world G1 experiments. Project website: https://venki1899.github.io/c-gmt/

--- Page 2 ---

compliant responses. Rather than asking the motion tracking policy itself to learn both tracking and compliance, we formulate compliance as a reference adaptation problem. This decomposition separates the two objectives: a forcerobust GMT based on [3] is responsible for faithfully executing the commanded motion and maintaining balance under disturbances, while a learned residual adapter modifies the reference motion to realize the desired compliance. To generate compliant reference motions for the downstream GMT, we construct an analytical teacher that observes the applied external force and uses a spring law and an inversekinematics (IK) solver to compute compliant adaptations. We can use the teacher and pretrained GMT to tune the desired response and verify its feasibility before training the adapter. To learn to estimate force and react appropriately at deployment time, we distill the compliant behavior into a residual adapter conditioned on proprioception that provides compliant references to the GMT. Our contributions are twofold: 1. We introduce Compliant GMT, a modular framework that unifies general motion tracking with adjustable wholebody compliance through residual reference adaptation. We propose a proprioceptive residual adapter distilled from a force-privileged teacher to generate compliant references. 2. Through simulation and real world experiments on the Unitree G1, we show that C-GMT matches both the compliance adherence of previous compliant single-motion tracking policies [4] and maintains tracking performance comparable to state-of-the-art GMTs. We further demonstrate C-GMT’s novel advantages of compliance during highly dynamic whole-body motions and coordinated whole-body compliant responses.

--- Page 3 ---

Compliant GMT decomposes compliant general motion tracking into two components: (i) a force-robust general motion tracker that faithfully executes reference motions under external loads, and (ii) a residual adapter that modifies the reference to produce a desired compliant response. This decomposition allows the desired compliant behavior to be specified independently of the motion-tracking objective.

TEACHER privileged — needs the true force and an IK solve Reference motion Compliance IK yield = F / k Force F · stiffness k Compliant reference Force-robust GMT frozen Teacher action aT STUDENT the same frozen tracker — no force sensor, no IK, nothing privileged Reference motion Proprioception + stiffness k Residual adapter + residual Δ Force-robust GMT frozen Student action aˢ Simulator

--- Page 4 ---

B. Compliant Teacher and Feasibility Tuning We next construct a privileged teacher that specifies the compliant response to be learned by the residual adapter. Given an external force applied to a link and a commanded stiffness, a spring law determines the desired position of the forced link relative to its original reference, pi,des = pi,ref + 1/Kcmd t Fi, (2) where Fi is the external force applied to link i and Kcmd t is the commanded stiffness. Then, an IK solver generates a corresponding whole-body reference. The IK objective prioritizes the desired pose of the forced link, followed by foot-placement and center-of-mass objectives for maintaining balance. Additional posture regularization keeps the solution close to the original reference motion and discourages unnecessary body movement. A critical benefit of this formulation is that the desired response can be evaluated for feasibility and style before training the residual adapter We execute the IK-generated references directly with the force-robust GMT and tune the IK objectives based on the resulting robot behavior.

1. The system can perform full-body compliant responses while maintaining general motion tracking capabilities. This allows humanoid robots to respond safely and robustly to interaction forces, mitigating the issue where existing general motion tracking policies are trained to be stiff, leading to unexpected and unsafe behaviors under disturbance.

2. The AI system can achieve a unified control objective by leveraging pre-trained models: "Our key insight is that we can achieve both competitive generality and compliance in a single policy by exploiting the capabilities of pretrained general motion trackers and introducing compliance through a residual adapter conditioned on a proprioception history."

3. The system can dynamically modulate its stiffness response based on external interaction: The framework allows the residual adapter to modify the reference motion to realize the desired compliance, enabling it to yield appropriately to external forces as specified by a commanded stiffness.

4. The AI system can enhance robustness during dynamic execution and high-level control: C-GMT demonstrates improved safety, robustness, and error tolerance in simulation and real-world G1 experiments, specifically showing that compliance improves robustness while executing high-level policy commands when facing unexpected forces during VLA rollouts.

5. The system can enable contact-rich teleoperation: C-GMT is capable of tasks where exact object and interaction dynamics are unknown and imperceivable to the operator, as it yield[s] to the contact while still closely tracking the commanded motion.

6. The system can provide coordinated whole-body compliance during disturbances: Unlike previous methods, C-GMT supports coordinated whole-body compliant responses when subjected to downward forces, allowing it to redistribute the disturbance through the rest of the body, whereas prior methods like GentleHumanoid eventually falls under a strong force.

7. The system can operate with reduced training time and higher adherence: By using a proprioceptive residual adapter distilled from a force-privileged teacher, C-GMT achieves compliance adherence comparable to single-motion policies while maintaining tracking performance comparable to state-of-the-art GMTs.

We propose Compliant GMT (C-GMT), a modular framework for full-body compliant responses in general motion tracking. As humanoid robots move into real world deployment, they not only need to perform any motion but also respond safely and robustly to interaction forces. Existing general motion tracking policies can imitate diverse motions but are trained to be stiff, leading to unexpected and unsafe behaviors under disturbance. Prior work in compliant humanoid control trades off compliance and generality. There exist examples of full-body compliant responses, but these are heavily curated to single-reference policies. On the other hand, general motion trackers (GMT) are trained to rigidly imitate massive motion capture datasets. Our key insight is to leverage the generality of pretrained GMTs for motion tracking while introducing compliance from a residual adapter that conditions on a history of proprioception. This enables our approach, C-GMT, to achieve both general motion tracking and whole-body compliant responses. Compared to state-of-the-art general motion trackers and single-motion compliant policies, C-GMT demonstrates competitive performance in both motion tracking accuracy and compliance adherence. We further demonstrate improved safety, robustness, and error tolerance in simulation and real-world G1 experiments.