Fangshuo Liao — accepted AI-conference papers

2025

Learning to Specialize: Joint Gating-Expert Training for Adaptive MoEs in Decentralized Settings

Yehya Farhat, Hamza ElMokhtar Shili, Fangshuo Liao, Chen Dun, Mirian Del Carmen Hipolito Garcia, Guoqing Zheng +4

NeurIPS 2025poster

Mixture-of-Experts (MoEs) achieve scalability by dynamically activating subsets of their components. Yet, understanding how expertise emerges through joint training of gating mechanisms and experts remains incomplete, especially in scenarios without clear task partitions. Motivated by inference cost…

Cited by 0 Source ↗Scholar ↗

2024

On the Error-Propagation of Inexact Hotelling's Deflation for Principal Component Analysis

Fangshuo Liao, Junhyung Lyle Kim, Cruz Barnum, Anastasios Kyrillidis

ICML 2024poster

Principal Component Analysis (PCA) aims to find subspaces spanned by the so-called *principal components* that best represent the variance in the dataset. The deflation method is a popular meta-algorithm that sequentially finds individual principal components, starting from the most important ones a…

Cited by 2 Source ↗PDF ↗Scholar ↗

2023

LOFT: Finding Lottery Tickets through Filter-wise Training

Qihan Wang, Chen Dun, Fangshuo Liao, Chris Jermaine, Anastasios Kyrillidis

AISTATS 2023poster

Recent work on the Lottery Ticket Hypothesis (LTH) shows that there exist “winning tickets” in large neural networks. These tickets represent “sparse” versions of the full model that can be trained independently to achieve comparable accuracy with respect to the full model. However, finding the winn…

Cited by 4 Source ↗PDF ↗Scholar ↗

2023

Scissorhands: Exploiting the Persistence of Importance Hypothesis for LLM KV Cache Compression at Test Time

Zichang Liu, Aditya Desai, Fangshuo Liao, Weitao Wang, Victor Xie, Zhaozhuo Xu +2

NeurIPS 2023poster

Large language models(LLMs) have sparked a new wave of exciting AI applications. Hosting these models at scale requires significant memory resources. One crucial memory bottleneck for the deployment stems from the context window. It is commonly recognized that model weights are memory hungry; howeve…

Cited by 200 Source ↗PDF ↗Scholar ↗

2023

Strong Lottery Ticket Hypothesis with $\varepsilon$–perturbation

Zheyang Xiong, Fangshuo Liao, Anastasios Kyrillidis

AISTATS 2023poster

The strong Lottery Ticket Hypothesis (LTH) (Ramanujan et al., 2019; Zhou et al., 2019) claims the existence of a subnetwork in a sufficiently large, randomly initialized neural network that approximates some target neural network without the need of training. We extend the theoretical guarantee of t…

Cited by 0 Source ↗PDF ↗Scholar ↗