Yunjia Zhang

MiniX: Mitigating Low-Rank Collapse and Attention Bottlenecks in Tabular Foundation Models

Recent tabular foundation models routinely match or surpass strong tree ensembles and specialized deep architectures, yet their numeric embeddings remain a bottleneck. We diagnose a low-rank collapse induced by the prevalent linear+ID scheme and introduce RaBEL, a compact Radial Basis Embedding Laye…