Liu Ruochen

Interpreting Positional Information in Perspective of Word Order

The attention mechanism is a powerful and effective method utilized in natural language processing. However, it has been observed that this method is insensitive to positional information. Although several studies have attempted to improve positional encoding and investigate the influence of word or…