Takumi Takada

Direct Metric Optimization for Image Captioning through Reward-Weighted Augmented Data Utilization

While image captioning is an essential field of vision language models (VLM), a lack of continuity between the learning objective and final performance metrics of VLMs complicates their training and optimization. Reinforcement learning (RL) can directly optimize such metrics, but it is accompanied b…