6 accepted papers
LLM-based multi-agent systems excel at planning, tool use, and role coordination, but their openness and interaction complexity also expose them to jailbreak and adversarial collaboration. Existing defenses fall into two lines: (i) self-verification that asks each agent to pre-filter unsafe instruct…
We show that Chain-of-Thought (CoT) enables a fixed single-layer transformer to efficiently approximate the training process of an $N$-layer feed-forward network in-context. Since FFNs are universal approximators, this result provides strong theoretical evidence for the expressive power of CoT. Spec…
Time series foundation models (TSFMs) are increasingly adopted as general-purpose time series learners. Although their training corpora are vast, they exclude peta-scale astronomical time series that exhibit unique challenges (e.g., irregular sampling, multiple variates, and heteroskedasticity) and …
We prove that with linear transformations, both (i) two-layer self-attention and (ii) one-layer self-attention followed by a softmax function are universal approximators for continuous sequence-to-sequence functions on compact domains. Our main technique is a new interpolation-based method for analy…
We study the in-context universal approximation and compositional generalization of softmax Transformers. We prove an in-context universality result: a fixed-weight softmax Transformer approximates a broad class of continuous sequence-to-sequence functions. Building on this universality, we establis…
We introduce an Outlier-Efficient Modern Hopfield Model (termed `OutEffHop`) and use it to address the outlier inefficiency problem of training gigantic transformer-based models. Our main contribution is a novel associative memory model facilitating _outlier-efficient_ associative memory retrievals.…