2026
MiniX: Mitigating Low-Rank Collapse and Attention Bottlenecks in Tabular Foundation Models
ICML 2026poster
Recent tabular foundation models routinely match or surpass strong tree ensembles and specialized deep architectures, yet their numeric embeddings remain a bottleneck. We diagnose a low-rank collapse induced by the prevalent linear+ID scheme and introduce RaBEL, a compact Radial Basis Embedding Laye…