Roman Castagné

MANTa: Efficient Gradient-Based Tokenization for End-to-End Robust Language Modeling

Static subword tokenization algorithms have been an essential component of recent works on language modeling. However, their static nature results in important flaws that degrade the models’ downstream performance and robustness. In this work, we propose MANTa, a Module for Adaptive Neural TokenizAt…