Zimu Yu

Causality-Induced Positional Encoding for Transformer-Based Representation Learning of Non-Sequential Features

Positional encoding is essential for supplementing transformer with positional information of tokens. Existing positional encoding methods demand predefined token/feature order, rendering them unsuitable for real-world data with non-sequential yet causally-related features. To address this limitatio…