Spark-0: Scaling Spatially Aligned Tactile Pretraining for Robot Dexterous Manipulation

arXiv:2610.22342 · cs.RO · Submitted 2026-10-15 · Read on arXiv

Listen

Radio episode about this paper

Transcript

Introduction to the show: ident: Robotics Radio. Generated commentary on the latest robotics and control papers.

Rosa: I'm Rosa.

Dev: And I'm Dev, here with Taro.

Rosa: Today's paper: "Spark-0: Scaling Spatially Aligned Tactile Pretraining for Robot Dexterous Manipulation".

Dev: The gist Spark-0, a spatially aligned vision-tactile-language-action (VTLA) model,

Rosa: First, who's behind it and why it matters.

Title and authors: Rosa: To get into the title, "Spark-zero: Scaling Spatially Aligned Tactile Pretraining for Robot Dexterous Manipulation," it’s pretty descriptive of what they did, focusing on scaling up that tactile pretraining specifically for dexterous manipulation <ref:2610.22342#pg1,Spark-0: Scaling Spatially Aligned Tactile Pretraining for Robot Dexterous Manipulation>.

Dev: They are tackling the difficulty in getting models to understand touch and spatial alignment consistently across different physical scenarios. It's about making sure the model learns the physics of contact reliably.

Taro: So, if we take "spatially aligned tactile pretraining," what does that actually mean for a robot learning to grab something?

Rosa: It means they are using touch data—specifically image-space contact maps derived from tactile gloves—to supervise the visual tokens during pretraining. This teaches the model exactly where contact happens in the visual space, even though there’s no actual touch at deployment one <ref:2610.22342#pg1>.

Dev: So even though raw touch never enters the vision-language backbone, it still influences how those visual representations are learned through that spatial alignment objective. It's a training signal they use.

Taro: That’s interesting because it suggests that the visual learning isn't just based on what the camera sees, but also on this synthetic understanding of where surfaces meet.

Rosa: Right, and in parallel to that, they have a tactile pathway that encodes regional force histories and three dee taxel states to condition the action expert directly one <ref:2610.22342#pg1>. This allows the action expert to learn how contact evolves dynamically without having raw touch data flowing through the main visual language model.

Dev: That’s a clever way to keep the backbone clean while still injecting critical physical information into the decision-making part of the system. It separates what you learn visually from how you decide what to do physically.

Taro: It’s like giving a student both a map and a set of instructions on how gravity affects them, instead of just showing them pictures and telling them to try it out one <ref:2610.22342#pg1>.

The paper's summary: Rosa: The core idea of this paper is that Spark-zero uses touch in two distinct ways during pretraining <ref:2610.22342#pg1>. First, it uses it as train-only supervision for visual tokens, and second, it uses tactile information to condition the action expert one <ref:2610.22342#pg1>.

Dev: That means they are using those contact maps derived from tactile gloves to guide what the model learns about the visual representations itself. They are teaching the model about where surfaces touch in a spatial sense.

Taro: So even though raw touch never enters the vision-language backbone, it still influences how those visual representations are learned through that spatial alignment objective. It’s a training signal they use to refine their understanding of three dee space and interaction <ref:2610.22342#pg1>.

Rosa: Right, and in parallel, a tactile pathway encodes regional force histories and three dee taxel states to condition the action expert directly one <ref:2610.22342#pg1>. This allows the action expert to learn how contact evolves dynamically without having raw touch data flowing through the main visual language model.

Dev: By doing that, they’re directly targeting those hard physical control problems by providing richer information to the action expert than vision alone can offer. It’s a direct input for the decision-making part.

Taro: So in plain terms, they’re giving the model a way to learn about physics—like compliance or incipient slip—that vision alone is missing, and that's where they see the real gains coming from one <ref:2610.22342#pg1>.

Rosa: That is exactly right. It’s not just about getting better visual priors; it't about getting better physical interaction understanding for manipulation, which is what matters for real robots.

Dev: So what does this mean practically? It suggests a separation between learning and deployment modes based on whether you have touch available or not, which is a huge practical point.

The paper's improvements: Rosa: The paper highlights several specific improvements they’ve made to the Spark-zero architecture, focusing on how they handle contact information that vision provides poorly one <ref:2610.22342#pg1>. They detail how they address these weaknesses in the system.

Dev: One key improvement is the train only spatial alignment which supervises visual tokens with contact regions derived from tactile gloves rendered into the head view. This teaches geometry alignment based on where things actually touch, not just relying on some idealized visual representation of a hand pose one <ref:2610.22342#pg1>.

Taro: So they are teaching geometry alignment based on where things actually touch, even when there's no tactile sensor available at deployment. That's a big step for robustness.

Rosa: That way, they are learning exactly where contact occurs even when there's no tactile sensor available at deployment. Then there’s geometry alignment which applies to every available view, and contact alignment which uses valid contact maps one <ref:2610.22342#pg1>.

Dev: And the tactile encoders learn through a layer-wise flow matching on samples with touch data to improve those contact alignments. They are using that touch data to refine the representations themselves.

Taro: So for a failure mode, if the visual input is noisy or misleading, this system has learned placeholders that can still guide the action expert gracefully because it's been trained on real contact information one <ref:2610.22342#pg1>.

Rosa: The tactile fusion adds a pathway that feeds only the action expert; raw touch never enters the VLM, and vision-only deployment replaces those tokens with learned placeholders. This makes vision-only deployment viable without requiring a tactile sensor at all.

Dev: That’s important because it means you don't necessarily need expensive hardware just to run the model in certain environments. You can use vision-only deployment when touch is missing one <ref:2610.22342#pg1>.

Conclusion: Rosa: To wrap up the Spark-zero paper, it uses touch both as train only supervision of visual tokens and as an input to the action expert one <ref:2610.22342#pg1>. This dual role makes it a very powerful VTLA model.

Dev: It achieves state of the art performance on both SparkArena tracks and ranks first on all eight aggregate metrics of the Ego Humanoid Manipulation Benchmark. That shows strong results in both vision-only simulated tasks and real-robot tasks with online touch two <ref:2610.22342#pg1>.

Taro: And I think the most important finding is that tactile pretraining on one thousand hours of HumanTouch raises vision-only success on Ego Humanoid by nineteen point two seven points, showing a significant gain in generalization to an unseen embodiment three <ref:2610.22342#pg3>.

Rosa: So in short, touch isn't just a policy input but also scalable supervision for dexterous visuomotor pretraining. It’s about using tactile data to make vision more capable in manipulation one <ref:2610.22342#pg1>.

Dev: This model provides a way to have one checkpoint that supports vision-only and tactile-conditioned deployment, which is really practical for real-world use cases where hardware might be missing one <ref:2610.22342#pg1>.

Taro: I think the final word is that touch isn't just a policy input but also scalable supervision for dexterous visuomotor pretraining. It shows how scalable data can improve the model’s ability to handle physical interaction when it doesn't have to be directly observed three <ref:2610.22342#pg1>.

Rosa: That’s what we’re hearing about Spark-zero and how it uses tactile data to make vision more capable in manipulation one <ref:2610.22342#pg1>.

Tianxing Chen, Wenwei Lin, Chuqiao Lyu, Peicheng Xiang, Junyuan Tang, Tian Nian, Zijian Cai, Kehe Ye, Chenze Yu, Wenxuan Zhu, §Eric J. Chen

1Xspark AI Institute of Technology University Hong Kong University of Science and Technology (GZ) Peking University

cs.RO

Submitted: 2026-10-15

Updated: 2026-10-15

Comments: Website: https://xsparkai.com/Spark-0

Code: https://github.com/XsparkAI/Spark-0

Project page: https://xsparkai.com/Spark-0

License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/

Importance score: 90/100

The gist: The gist Spark-0, a spatially aligned vision-tactile-language-action (VTLA) model, utilizes touch in two complementary ways—as train-only supervision for visual tokens and as an input to condition

Key concepts

VTLA Model: This is a type of AI model that combines vision (seeing), tactile sensing (touch), language (understanding instructions), and action (doing). Spark-0 specifically integrates these four modalities. It allows the robot to learn complex tasks by processing visual information, understanding language, using touch data for supervision during training, and generating physical actions.
Train-only Spatial Alignment: This is a method used during pretraining where contact maps generated from tactile gloves are used to supervise the visual tokens. This means the model learns to align its visual representation with where physical contact actually occurs. This helps the model understand spatial relationships related to touching, even though raw touch data is not used when it's deployed.
Tactile Pathway: This is a specific pathway within Spark-0 that encodes regional force histories and 3D tactile states. This pathway does not directly influence the main vision-language backbone but instead feeds information specifically to condition the action expert. It allows the model to use detailed, localized touch information to make better decisions about what physical actions to take.
Scalable Supervision: The paper demonstrates that using large amounts of high-quality tactile data (like 1,000 hours of HumanTouch) as supervision during pretraining can significantly boost the performance of vision-only policies. This shows that touch isn't just a final input for control, but it can be used as a powerful, scalable way to teach the robot how to handle contact-rich tasks from scratch.

Terminology

Benchmark — 1× in this paper · explained in 2 episodes

Summary

The gist Spark-0, a spatially aligned vision-tactile-language-action (VTLA) model, utilizes touch in two complementary ways—as train-only supervision for visual tokens and as an input to condition the action expert—to achieve state-of-the-art performance in dexterous manipulation benchmarks

How it works

Spark-0 employs a dual role for touch during pretraining, supporting both vision-only and tactile-conditioned deployment from a single checkpoint During pretraining, image-space contact maps derived from tactile gloves, together with features from a frozen 3D geometry teacher, supervise the visual tokens In parallel, a tactile pathway encodes regional force histories and 3D taxel states to condition the action expert Raw touch never enters the vision-language backbone, so one checkpoint supports both vision-only and tactile-conditioned deployment

Data and Pretraining

The model is pretrained on a heterogeneous mixture where HumanTouch provides full-hand contact supervision from human demonstrations This dataset consists of 1,000 hours of manipulation data across more than 100 tasks and 32 operators Spark-0 is pretrained on HumanTouch together with egocentric video, robot data, and simulation The pretraining mixture includes HumanTouch (1,000 hours), EgoDex egocentric video [29], real-robot data align the model with the target embodiment and add robot-task diversity [3] and domain-randomized simulation adds visual variation

Architecture

The Spark-0 architecture pairs Qwen3-VL-4B with a layerwise flow-matching action expert so that action generation can draw on the full visual hierarchy The VLM produces hidden states h(l) [4] and learnable action queries appended to the VLM sequence summarize the visual-language context The action expert has L = 36 DiT blocks of width 1024, matching the VLM depth Even blocks cross-attend to the projected hidden state of the corresponding VLM layer, including 64 action queries Odd blocks perform the joint self-attention of Section 4.3

Supervision and Fusion

Spark-0 addresses contact-rich dexterous control by addressing two kinds of information that vision provides poorly: where contact occurs and how it evolves Train-only spatial alignment supervises visual tokens with contact regions (Section 4.2) Geometry alignment applies to every available view, contact alignment to valid contact maps, and the tactile encoders learn through Lact on samples with touch Contact alignment renders the mesh into the head view, and a lightweight head predicts contact logits from the head-view tokens Tactile fusion adds a tactile pathway that feeds only the action expert; raw touch never enters the VLM, and vision-only deployment replaces tactile tokens with learned placeholders

Evaluation

SparkArena evaluates both vision-only simulated tasks and tactile-conditioned real-robot tasks on one dexterous embodiment SparkArena provides one track for each, testing vision-only dexterity and contact-rich manipulation with online touch Spark-0 achieves the highest mean success rate on both SparkArena tracks and ranks first on all eight aggregate metrics of the Ego Humanoid Manipulation Benchmark Tactile pretraining on 1,000 hours of HumanTouch raises vision-only success on Ego Humanoid from 52.5% to 71.7%

Key Findings

First, without touch at deployment, Spark-0 outperforms all baselines on both the target embodiment and an unseen humanoid Second, with online touch, it exceeds the strongest baseline, itself a tactile policy, by 16.00 points in success on contact-rich real-robot tasks Third, the cross-embodiment gain stems from tactile pretraining and is most evident at full scale: 1,000 hours of HumanTouch raise vision-only success on Ego Humanoid by 19.27 points This indicates that touch is not only a policy input but also scalable supervision for dexterous visuomotor pretraining

Conclusion

Spark-0 presents a VTLA model that uses touch both as train-only supervision of visual tokens and as an input to the action expert, so a single checkpoint supports vision-only and tactile-conditioned deployment It achieves the highest mean success rate on both SparkArena tracks and ranks first on all Ego Humanoid aggregate metrics Tactile pretraining on 1,000 hours of HumanTouch yields its largest gain on the unseen Ego Humanoid embodiment under vision-only deployment, indicating that touch is not only a policy input but also scalable supervision for dexterous visuomotor pretraining

References

[1] P. Intelligence, K. Black, N. Brown, J. Darpinian, K. Dhabalia, D. Driess, A. Esmail, M. Equi, C. Finn, N. Fusai, M. Y Galliker, D Ghosh | 1

[2] NVIDIA:, J Bjorck:, F Castaneda:, N Cherniadev:, X Da:, R Ding:, L J Fan:, Y Fang:, D Fox: | 2

[3] S Community.: StarVLA: A Lego-like Codebase for Vision-Language-Action Model Developing | 3

[4] D Niu, Z Liu, Z Wang, B Shao, Z H Yin, A Pai, Y Sharma, S Saravalle, R Zheng: | 4

[5] C Yuan, Z Zhang, M Zhou: FTP-1: A Generalist Foundation Tactile Policy Across Tactile Sensors for Contact-Rich Manipulation | 5

[6] C Zhang, P Cai, Z Xi, H Yuan, H Luo: HumanCentric Transferable Tactile Pre-Training for Dexterous Robotic Manipulation | 6

[7] N Team and F T Team: N0-VTLA: Scaling Vision-Tactile-Language-Action Model with Latent Tactile Tokens | 7

[8] R Yang, Q Yu, Y Wu, R Yan, B Li, A C Cheng, X Zou: EgoVLA: Learning Vision-Language-Action Models from Egocentric Human Videos | 8

[9] W Yuan, S Dong, E H Adelson: HumanTouch contains 1000 hours of manipulation data spanning more than 100 tasks and 32 operators | 9

[10] M Lambeta, P W Chou, S Tian, B Yang: DIGIT: A Novel Design for a Low-Cost Compact High-Resolution Tactile Sensor with Application to In-Hand Manipulation | 10

[11] Y R Song, J Li, R Fu, D Murphy: OPENTOUCH: Bringing FullHand Touch to Real-World Interaction | 11

[12] Y Zheng, J Peng, W Li, Y Zheng: World In Your Hands: A Large-Scale and Open-Source Ecosystem for Learning Human-Centric Manipulation in the Wild | 12

[13] L Wu, C Yu, J Ren: FreeTacMan: Robotfree Visuo-Tactile Data Collection System for Contact-rich Manipulation | 13

[14] X Zhu, B Huang, Y Li: Touch in the Wild: Learning Fine-Grained Manipulation with a Portable Visuo-Tactile Gripper | 14

[15] B Chen, W Wan, T Chen, X Guo: UniVTAC: A Unified Simulation Platform for VisuoTactile Manipulation Data Generation, Learning, and Benchmarking | 15

[16] J Zhou, Z Gao, F Hong: TouchAnything: A Dataset and Framework for Bimanual Tactile Estimation from Egocentric Video | 16

[17] D Zhou, J Lu, J Lin: TouchSight: Bare-Handed Tactile Prediction from Egocentric Video via Generative Visual Augmentation | 17

[18] C Liang, Y Lai, C Lyu: BiView-Touch: Learning Bimanual Tactile Representations by Cross-Hand Completion | 18

[19] N Team and F T Team: N0-Foundation: Towards the Age of Tactile Intelligence | 20

[20] A Brohan, N Brown, J Carbajal, Y Chebotar, X Chen: RT-2: Vision-Language-Action Models Transfer Web Knowledge to Robotic Control | 21

[21] M J Kim, K Pertsch, S Karamcheti: OpenVLA: An Open-Source Vision-Language-Action Model | 21

[22] T Z Zhao, V Kumar, S Levine, C Finn: Learning Fine-Grained Bimanual Manipulation with Low-Cost Hardware | 23

[23] C Chi, Z Xu, S Feng: Diffusion Policy: Visuomotor Policy Learning via Action Diffusion | 24

[24] K Black, N Brown, D Driess, A Esmail: π0: A Vision-Language-Action Flow Model for General Robot Control 24

[25] D Kim, H Jang, M Koo:

Improvements for AI systems

textbf Tactile Supervision for Visual Tokens in Pretraining (Train-Only Objective): This allows Spark-0 to supervise visual tokens with contact regions using glove-derived contact maps rendered into the head view, enabling the model to learn where contact occurs even when no tactile sensor is available at deployment, as it uses this objective during pretraining and removes it at inference.
textbf Contact-Conditioned Action Expert: This enables Spark-0 to condition the action expert on contact dynamics by using a tactile pathway that encodes regional force histories and 3D taxel states, allowing the model to learn how contact evolves, even when raw touch never enters the vision-language backbone.
textbf Unified Checkpoint Deployment: This allows a single checkpoint to support both deployment modes, as the architecture is designed so that raw touch never enters the vision-language backbone, so one checkpoint serves both vision-only and tactile-conditioned deployment.
textbf Scalable HumanTouch Data Collection: The creation of HumanTouch, a 1,000-hour dataset of full-hand tactile human manipulation, provides the necessary scale for training Spark-0 to achieve high performance by allowing it to benefit from scalable supervision for dexterous visuomotor pretraining.
textbf Dual-Role Evaluation Benchmark: The release of SparkArena, which pairs vision-only simulated tasks with tactile-conditioned real-robot tasks on one dexterous embodiment, allows researchers to test both vision-only dexterity and contact-rich manipulation on the same robot, addressing the gap in existing benchmarks.
textbf Cross-Embodiment Generalization via Tactile Pretraining: The results show that 1,000 hours of HumanTouch raise vision-only success on Ego Humanoid by 19.27 points, demonstrating that tactile pretraining and is most evident at full scale for improving generalization to unseen embodiments.
textbf Real-Robot Contact-Rich Control: Spark-0 achieves a 52.00% success and a 62.80 completion score on SparkArena-Real, exceeding the strongest baseline by 16.00 and 15.00 points, meaning it can perform contact-rich manipulation where contact state is poorly observable from vision online.