CCIM: Cross-modal Cross-lingual Interactive Image Translation
Cong MA, Yaping Zhang, Mei Tu, Yang Zhao, Yu Zhou, Chengqing Zong
Abstract
Text image machine translation (TIMT) which translates source language text images into target language texts has attracted intensive attention in recent years. Although the end-to-end TIMT model directly generates target translation from encoded text image features with an efficient architecture, it lacks the recognized source language information resulting in a decrease in translation performance. In this paper, we propose a novel Cross-modal Cross-lingual Interactive Model (CCIM) to incorporate source language information by synchronously generating source language and target language results through an interactive attention mechanism between two language decoders. Extensive experimental results have shown the interactive decoder significantly outperforms end-to-end TIMT models and has faster decoding speed with smaller model size than cascade models.
BibTeX
@inproceedings{
ma2023ccim,
title={{CCIM}: Cross-modal Cross-lingual Interactive Image Translation},
author={Cong MA and Yaping Zhang and Mei Tu and Yang Zhao and Yu Zhou and Chengqing Zong},
booktitle={The 2023 Conference on Empirical Methods in Natural Language Processing},
year={2023},
url={https://openreview.net/forum?id=iCNoSVJl2y}
}