Dianyi Wang

Activating Distributed Visual Region within LLMs for Efficient and Effective Vision-Language Training and Inference

Large Vision-Language Models (LVLMs) typically learn visual capacity through visual instruction tuning, involving updates to both a projector and their LLM backbones. Inspired by the concept of a visual region in the human brain, we investigate the existence of an analogous visual region within LLMs…