Tohru Nitta

Quaternion Self-Attention with Shared Scores

Quaternion Neural Networks are parameter-efficient and model multidimensional dependencies by representing four related features as a single entity. However, existing quaternion self-attention computes component-wise scores and applies independent softmax operations to each component, which increase…