VisionPsy-Nano: Improving Accuracy, Efficiency, and Reliability in On-Device Vision-Language Models

arXiv:2609.31746 · cs.CV · Submitted 2026-09-23 · Read on arXiv

cs.CV

Submitted: 2026-09-23

Updated: 2026-09-23

Code: https://github.com/huggingface/nanoVLM

License: http://creativecommons.org/licenses/by/4.0/

The gist: Sub-billion-parameter Vision-Language Models are increasingly viable for on-device deployment, yet compact model size alone does not guarantee usability.

Terminology

Abstract

Sub-billion-parameter Vision-Language Models are increasingly viable for on-device deployment, yet compact model size alone does not guarantee usability. On a phone, such a model can still require more than two minutes to produce its first token. On-device usability depends on three axes: accuracy, efficiency, and behavioral reliability; standard benchmarks miss the third, with answers too short to expose doom loops and prompts too benign to probe adversarial safety. We introduce a diagnosis-driven post-training recipe in which a teacher VLM stress-tests the student, uncovers failure modes beyond human priors, and converts them into targeted supervision and preference alignment, supplementing generic data scaling with failure-driven optimization. Coupled with two visual-token policies, the recipe yields two accuracy-efficiency variants with improved behavioral reliability. attains a 62.3 normalized average over 17 benchmarks, the highest among openly released about 0.5B models, +7.4 over its base at identical architecture and token budget, with doom-loop rates at or below the strongest baseline's. retains 61.4 while cutting warm time-to-first-token on a Pixel 9 from 138,s to 6.1,s (23 times). By jointly addressing all three axes, we move compact VLMs toward practical on-device usability.

Sources

Related papers