Illuminating Unified Multimodal Model for Free-form Interleaved Text-Image Generation
cs.CV
Submitted: 2026-06-29
Updated: 2026-06-29
Comments: Accepted by ECCV2026
DOI: 10.1007/978-3-032-37032-7_29
Code: https://github.com/ChonghuinanWang/ILLUME-X
License: http://creativecommons.org/licenses/by/4.0/
The gist: The advancement of generative AI models capable of producing text and image marks a critical step forward in the realm of multimodal intelligence, particularly for tasks involving the interleaving of
Terminology
Abstract
The advancement of generative AI models capable of producing text and image marks a critical step forward in the realm of multimodal intelligence, particularly for tasks involving the interleaving of both modalities. To advance this intelligence to the next stage, it is crucial for models to autonomously generate free-form interleaved text-image sequences. In this paper, we introduce ILLUME-X, an advanced unified multimodal paradigm that enables high-quality, free-form interleaved text-image generation by improving multimodal data efficiency and stabilizing the multimodal training process. ILLUME-X comprises three key components: (i) an expanded training data pipeline optimized for interleaved text-image generation, (ii) a progressive training strategy with self-adaptive objectives for free-length multimodal token sequences, and (iii) an objective and comprehensive evaluation method ILScore for interleaved text-image sequences. Notably, our ILLUME-X outperforms previous unified models across multiple interleaved text-image generation tasks like style transfer, image decomposition and storytelling.
Sources
- HunyuanImage 3.0 Technical Report
- Interleaved Scene Graphs for Interleaved Text-and-Image Generation Assessment
- BLIP3-o: A Family of Fully Open Unified Multimodal Models-Architecture, Training and Dataset
- ANOLE: An Open, Autoregressive, Native Large Multimodal Models for Interleaved Image-Text Generation
- WEAVE: Unleashing and Benchmarking the In-context Interleaved Comprehension and Generation
- Gemini 2.5: Pushing the Frontier with Advanced Reasoning, Multimodality, Long Context, and Next Generation Agentic Capabilities
- Qwen3-VL Technical Report
- Qwen2.5-VL Technical Report
- Janus-Pro: Unified Multimodal Understanding and Generation with Data and Model Scaling
- Emu3.5: Native Multimodal Models are World Learners
- Making LLaMA SEE and Draw with SEED Tokenizer
- Zebra-CoT: A Dataset for Interleaved Vision Language Reasoning
- Lavida-O: Elastic Large Masked Diffusion Models for Unified Multimodal Understanding and Generation
- Emerging Properties in Unified Multimodal Pretraining
- EMMA: Efficient Multimodal Understanding, Generation, and Editing with a Unified Architecture
- ILLUME+: Illuminating Unified MLLM with Dual Visual Tokenization and Diffusion Refinement
- UniWorld-V1: High-Resolution Semantic Encoders for Unified Visual Understanding and Generation
- Muddit: Liberating Generation Beyond Text-to-Image with a Unified Discrete Diffusion Model
- OpenAI GPT-5 System Card
- Gemini: A Family of Highly Capable Multimodal Models
Related papers
- Loss Knows Best: Detecting Annotation Errors in Videos via Loss Trajectories
- AnchorWeave: World-Consistent Video Generation with Retrieved Local Spatial Memories
- Benchmarking the Robustness of Foundation Models for Mammography under Domain Shift
- MambaX-Net: Dual-Input Mamba-Enhanced Cross-Attention Network for Longitudinal MRI Segmentation
- TeleOCR: Navigating Document Parsing Across Digital and Camera-Captured Documents
- A Survey on Efficient Vision-Language-Action Models