AIConfPaper
← Search

Chun-Liang Li

43 accepted papers

2026

AMusE: Audio-Visual Benchmark and Alignment Framework for Agentic Multi-Speaker Understanding

Sanjoy Chowdhury, Karren D Yang, Xudong Liu, Fartash Faghri, Pavan Kumar Anasosalu Vasu, Oncel Tuzel +3
CVPR 2026

Recent multimodal large language models (MLLMs) such as GPT-4o and Qwen3-Omni show strong perception but struggle in multi-speaker, dialogue-centric settings that demand agentic reasoning, tracking who speaks, maintaining roles, and grounding events across time. These scenarios are central to multim

Cited by 0Source ↗Scholar ↗
2026

Rethinking JEPA: Compute‑Efficient Video Self-Supervised Learning with Frozen Teachers

Xianhang Li, Chen Huang, Chun-Liang Li, Eran Malach, Joshua M. Susskind, Vimal Thilak +1
ICLR 2026poster

Video Joint Embedding Predictive Architectures (V‑JEPA) learn generalizable off-the-shelf video representations by predicting masked regions in latent space with an exponential moving average (EMA)‑updated teacher. While EMA prevents representation collapse, it complicates scalable model selection a…

Cited by 0Source ↗Scholar ↗
2026

TFRBench: A Reasoning Benchmark for Evaluating Forecasting Systems

Md Atik Ahamed, Mihir Parmar, Palash Goyal, Yiwen Song, Long T. Le, Qiang (Shaun) Cheng +4
ICML 2026poster

We introduce TFRBench, the first benchmark designed to evaluate the reasoning capabilities of forecasting systems. Traditionally, time-series forecasting has been evaluated solely on numerical accuracy, treating foundation models as "black boxes." Unlike existing benchmarks, TFRBench provides a prot…

Cited by 0Source ↗Scholar ↗
2026

TrajTok: Learning Trajectory Tokens Enhances Video Understanding

Chenhao Zheng, Jieyu Zhang, Jianing Zhang, Weikai Huang, Ashutosh Kumar, Quan Kong +3
CVPR 2026

Tokenization in video models, typically through patchification, generates an excessive and redundant number of tokens. This severely limits video efficiency and scalability. While the recent trajectory-based tokenizers offer a promising solution by decoupling video duration from token count, they re

Cited by 0Source ↗code ↗Scholar ↗
2025

FastVLM: Efficient Vision Encoding for Vision Language Models

Pavan Kumar Anasosalu Vasu, Fartash Faghri, Chun-Liang Li, Cem Koc, Nate True, Albert Antony +5
CVPR 2025poster

Vision Language Models (VLMs) like LLaVA encode images into tokens aligned to the word embedding space of the LLM decoder. Scaling input image resolution is essential for improving performance, especially in text-rich image understanding tasks. However, popular visual encoders such as CLIP-pretraine…

Cited by 7Source ↗PDF ↗code ↗Scholar ↗
2024

Chain-of-Table: Evolving Tables in the Reasoning Chain for Table Understanding

Zilong Wang, Hao Zhang, Chun-Liang Li, Julian Martin Eisenschlos, Vincent Perot, Zifeng Wang +5
ICLR 2024poster

Table-based reasoning with large language models (LLMs) is a promising direction to tackle many table understanding tasks, such as table-based question answering and fact verification. Compared with generic reasoning, table-based reasoning requires the extraction of underlying semantics from both fr…

Cited by 107Source ↗PDF ↗Scholar ↗
2024

CodecLM: Aligning Language Models with Tailored Synthetic Data

Zifeng Wang, Chun-Liang Li, Vincent Perot, Long Le, Jin Miao, Zizhao Zhang +2
NAACL 2024findings

Instruction tuning has emerged as the key in aligning large language models (LLMs) with specific task instructions, thereby mitigating the discrepancy between the next-token prediction objective and users’ actual goals. To reduce the labor and time cost to collect or annotate data by humans, researc…

Cited by 19Source ↗PDF ↗code ↗Scholar ↗
2024

Dataset Decomposition: Faster LLM Training with Variable Sequence Length Curriculum

Hadi Pouransari, Chun-Liang Li, Jen-Hao Rick Chang, Pavan Kumar Anasosalu Vasu, Cem Koc, Vaishaal Shankar +1
NeurIPS 2024poster

Large language models (LLMs) are commonly trained on datasets consisting of fixed-length token sequences. These datasets are created by randomly concatenating documents of various lengths and then chunking them into sequences of a predetermined target length (concat-and-chunk). Recent attention impl…

Cited by 10Source ↗PDF ↗code ↗Scholar ↗
2024

Found in the middle: Calibrating Positional Attention Bias Improves Long Context Utilization

Cheng-Yu Hsieh, Yung-Sung Chuang, Chun-Liang Li, Zifeng Wang, Long Le, Abhishek Kumar +5
ACL 2024findings

Large language models (LLMs), even when specifically trained to process long input contexts, struggle to capture relevant information located in the middle of their input. This phenomenon has been known as the lost-in-the-middle problem. In this work, we make three contributions. First, we set out t…

Cited by 18Source ↗PDF ↗code ↗Scholar ↗
2024

MUSCLE: A Model Update Strategy for Compatible LLM Evolution

Jessica Maria Echterhoff, Fartash Faghri, Raviteja Vemulapalli, Ting-Yao Hu, Chun-Liang Li, Oncel Tuzel +1
EMNLP 2024finding

Large Language Models (LLMs) are regularly updated to enhance performance, typically through changes in data or architecture. Within the update process, developers often prioritize improving overall performance metrics, paying less attention to maintaining compatibility with earlier model versions.…

Cited by 4Source ↗PDF ↗Scholar ↗
2024

The Unmet Promise of Synthetic Training Images: Using Retrieved Real Images Performs Better

Scott Geng, Cheng-Yu Hsieh, Vivek Ramanujan, Matthew Wallingford, Chun-Liang Li, Pang Wei Koh +1
NeurIPS 2024poster

Generative text-to-image models enable us to synthesize unlimited amounts of images in a controllable manner, spurring many recent efforts to train vision models with synthetic data. However, every synthetic image ultimately originates from the upstream data used to train the generator. Does the int…

Cited by 4Source ↗PDF ↗code ↗Scholar ↗
2023

Distilling Step-by-Step! Outperforming Larger Language Models with Less Training Data and Smaller Model Sizes

Cheng-Yu Hsieh, Chun-Liang Li, Chih-kuan Yeh, Hootan Nakhost, Yasuhisa Fujii, Alex Ratner +3
ACL 2023findings

Deploying large language models (LLMs) is challenging because they are memory inefficient and compute-intensive for practical applications. In reaction, researchers train smaller task-specific models by either finetuning with human labels or distilling using LLM-generated labels. However, finetuning…

Cited by 501Source ↗PDF ↗code ↗Scholar ↗
2023

FormNetV2: Multimodal Graph Contrastive Learning for Form Document Information Extraction

Chen-Yu Lee, Chun-Liang Li, Hao Zhang, Timothy Dozat, Vincent Perot, Guolong Su +10
ACL 2023long

The recent advent of self-supervised pre-training techniques has led to a surge in the use of multimodal learning in form document understanding. However, existing approaches that extend the mask language modeling to other modalities require careful multi-task tuning, complex reconstruction target d…

Cited by 12Source ↗PDF ↗code ↗Scholar ↗
2023

Hyperbolic Contrastive Learning for Visual Representations Beyond Objects

Songwei Ge, Shlok Mishra, Simon Kornblith, Chun-Liang Li, David Jacobs
CVPR 2023poster

Although self-/un-supervised methods have led to rapid progress in visual representation learning, these methods generally treat objects and scenes using the same lens. In this paper, we focus on learning representations of objects and scenes that preserve the structure among them. Motivated by the…

Cited by 70Source ↗PDF ↗code ↗Scholar ↗
2023

Neural Spline Search for Quantile Probabilistic Modeling

Ruoxi Sun, Chun-Liang Li, Sercan Ö. Arik, Michael W. Dusenberry, Chen-Yu Lee, Tomas Pfister
AAAI 2023technical

Accurate estimation of output quantiles is crucial in many use cases, where it is desired to model the range of possibility. Modeling target distribution at arbitrary quantile levels and at arbitrary input attribute levels are important to offer a comprehensive picture of the data, and requires the…

Cited by 5Source ↗PDF ↗Scholar ↗
2023

Pic2Word: Mapping Pictures to Words for Zero-Shot Composed Image Retrieval

Kuniaki Saito, Kihyuk Sohn, Xiang Zhang, Chun-Liang Li, Chen-Yu Lee, Kate Saenko +1
CVPR 2023poster

In Composed Image Retrieval (CIR), a user combines a query image with text to describe their intended target. Existing methods rely on supervised learning of CIR models using labeled triplets consisting of the query image, text specification, and the target image. Labeling such triplets is expensive…

Cited by 137Source ↗PDF ↗code ↗Scholar ↗
2023

Prefix Conditioning Unifies Language and Label Supervision

Kuniaki Saito, Kihyuk Sohn, Xiang Zhang, Chun-Liang Li, Chen-Yu Lee, Kate Saenko +1
CVPR 2023poster

Pretraining visual models on web-scale image-caption datasets has recently emerged as a powerful alternative to traditional pretraining on image classification data. Image-caption datasets are more "open-domain", containing broader scene types and vocabulary words, and result in models that have str…

Cited by 15Source ↗PDF ↗Scholar ↗
2022

DISSECT: Disentangled Simultaneous Explanations via Concept Traversals

Asma Ghandeharioun, Been Kim, Chun-Liang Li, Brendan Jou, Brian Eoff, Rosalind Picard
ICLR 2022poster

Explaining deep learning model inferences is a promising venue for scientific understanding, improving safety, uncovering hidden biases, evaluating fairness, and beyond, as argued by many scholars. One of the principal benefits of counterfactual explanations is allowing users to explore "what-if" sc…

Cited by 64Source ↗PDF ↗code ↗Scholar ↗
2022

Decoupling Local and Global Representations of Time Series

Sana Tonekaboni, Chun-Liang Li, Sercan O. Arik, Anna Goldenberg, Tomas Pfister
AISTATS 2022poster

Real-world time series data are often generated from several sources of variation. Learning representations that capture the factors contributing to this variability enables better understanding of the data via its underlying generative process and can lead to improvements in performance on downstre…

Cited by 29Source ↗PDF ↗code ↗Scholar ↗
2022

FormNet: Structural Encoding beyond Sequential Modeling in Form Document Information Extraction

Chen-Yu Lee, Chun-Liang Li, Timothy Dozat, Vincent Perot, Guolong Su, Nan Hua +4
ACL 2022long

Sequence modeling has demonstrated state-of-the-art performance on natural language and document understanding tasks. However, it is challenging to correctly serialize tokens in form-like documents in practice due to their variety of layout patterns. We propose FormNet, a structure-aware sequence mo…

Cited by 87Source ↗PDF ↗Scholar ↗
2022

Learning Instance-Specific Adaptation for Cross-Domain Segmentation

Yuliang Zou, Zizhao Zhang, Chun-Liang Li, Han Zhang, Tomas Pfister, Jia-Bin Huang
ECCV 2022poster

"We propose a test-time adaptation method for cross-domain image segmentation. Our method is simple: Given a new unseen instance at the test time, we adapt a pre-trained model by conducting instance-specific BatchNorm (statistics) calibration. Our approach has two core components. First, we replace…

Cited by 16Source ↗PDF ↗Scholar ↗
2021

$i$-Mix: A Domain-Agnostic Strategy for Contrastive Representation Learning

Kibok Lee, Yian Zhu, Kihyuk Sohn, Chun-Liang Li, Jinwoo Shin, Honglak Lee
ICLR 2021poster

Contrastive representation learning has shown to be effective to learn representations from unlabeled data. However, much progress has been made in vision domains relying on data augmentations carefully designed using domain knowledge. In this work, we propose i-Mix, a simple yet effective domain-ag…

Cited by 153Source ↗PDF ↗code ↗Scholar ↗
2021

CutPaste: Self-Supervised Learning for Anomaly Detection and Localization

Chun-Liang Li, Kihyuk Sohn, Jinsung Yoon, Tomas Pfister
CVPR 2021poster

We aim at constructing a high performance model for defect detection that detects unknown anomalous patterns of an image without anomalous data. To this end, we propose a two-stage framework for building anomaly detectors using normal training data only. We first learn self-supervised deep represent…

Cited by 1095PDF ↗Scholar ↗
2021

Learning and Evaluating Representations for Deep One-Class Classification

Kihyuk Sohn, Chun-Liang Li, Jinsung Yoon, Minho Jin, Tomas Pfister
ICLR 2021poster

We present a two-stage framework for deep one-class classification. We first learn self-supervised representations from one-class data, and then build one-class classifiers on learned representations. The framework not only allows to learn better representations, but also permits building one-class…

Cited by 271Source ↗PDF ↗code ↗Scholar ↗
2021

Object-aware Contrastive Learning for Debiased Scene Representation

Sangwoo Mo, Hyunwoo Kang, Kihyuk Sohn, Chun-Liang Li, Jinwoo Shin
NeurIPS 2021poster

Contrastive self-supervised learning has shown impressive results in learning visual representations from unlabeled images by enforcing invariance against different data augmentations. However, the learned representations are often contextually biased to the spurious scene correlations of different…

Cited by 51Source ↗PDF ↗code ↗Scholar ↗
2021

PseudoSeg: Designing Pseudo Labels for Semantic Segmentation

Yuliang Zou, Zizhao Zhang, Han Zhang, Chun-Liang Li, Xiao Bian, Jia-Bin Huang +1
ICLR 2021poster

Recent advances in semi-supervised learning (SSL) demonstrate that a combination of consistency regularization and pseudo-labeling can effectively improve image classification accuracy in the low-data regime. Compared to classification, semantic segmentation tasks require much more intensive labelin…

Cited by 413Source ↗PDF ↗code ↗Scholar ↗
2021

ROPE: Reading Order Equivariant Positional Encoding for Graph-based Document Information Extraction

Chen-Yu Lee, Chun-Liang Li, Chu Wang, Renshen Wang, Yasuhisa Fujii, Siyang Qin +2
ACL 2021short

Natural reading orders of words are crucial for information extraction from form-like documents. Despite recent advances in Graph Convolutional Networks (GCNs) on modeling spatial layout patterns of documents, they have limited ability to capture reading orders of given word-level node representatio…

Cited by 30Source ↗PDF ↗Scholar ↗
2021

Robust Contrastive Learning Using Negative Samples with Diminished Semantics

Songwei Ge, Shlok Kumar Mishra, Chun-Liang Li, Haohan Wang, David Jacobs
NeurIPS 2021poster

Unsupervised learning has recently made exceptional progress because of the development of more effective contrastive learning methods. However, CNNs are prone to depend on low-level features that humans deem non-semantic. This dependency has been conjectured to induce a lack of robustness to image…

Cited by 86Source ↗PDF ↗code ↗Scholar ↗
2021

Unsupervised program synthesis for images by sampling without replacement

Chenghui Zhou, Chun-Liang Li, Barnabás Póczos
UAI 2021poster

Program synthesis has emerged as a successful approach to the image parsing task. Most prior works rely on a two-step scheme involving supervised pretraining of a Seq2Seq model with synthetic programs followed by reinforcement learning (RL) for fine-tuning with real reference images. Fully unsupervi…

Cited by 3Source ↗PDF ↗Scholar ↗
2020

FixMatch: Simplifying Semi-Supervised Learning with Consistency and Confidence

Kihyuk Sohn, David Berthelot, Nicholas Carlini, Zizhao Zhang, Han Zhang, Colin A Raffel +3
NeurIPS 2020poster

Semi-supervised learning (SSL) provides an effective means of leveraging unlabeled data to improve a model’s performance. This domain has seen fast progress recently, at the cost of requiring more complex methods. In this paper we propose FixMatch, an algorithm that is a significant simplification o…

Cited by 4608Source ↗PDF ↗code ↗Scholar ↗
2020

Interpretable Sequence Learning for Covid-19 Forecasting

Sercan Arik, Chun-Liang Li, Jinsung Yoon, Rajarishi Sinha, Arkady Epshteyn, Long Le +8
NeurIPS 2020spotlight

We propose a novel approach that integrates machine learning into compartmental disease modeling (e.g., SEIR) to predict the progression of COVID-19. Our model is explainable by design as it explicitly shows how different compartments evolve and it uses interpretable encoders to incorporate covariat…

Cited by 106Source ↗PDF ↗Scholar ↗
2020

On Completeness-aware Concept-Based Explanations in Deep Neural Networks

Chih-Kuan Yeh, Been Kim, Sercan Arik, Chun-Liang Li, Tomas Pfister, Pradeep K. Ravikumar
NeurIPS 2020poster

Human explanations of high-level decisions are often expressed in terms of key concepts the decisions are based on. In this paper, we study such concept-based explainability for Deep Neural Networks (DNNs). First, we define the notion of \emph{completeness}, which quantifies how sufficient a particu…

Cited by 384Source ↗PDF ↗code ↗Scholar ↗
2019

Beyond Pixel Norm-Balls: Parametric Adversaries using an Analytically Differentiable Renderer

Hsueh-Ti Derek Liu, Michael Tao, Chun-Liang Li, Derek Nowrouzezahrai, Alec Jacobson
ICLR 2019poster

Many machine learning image classifiers are vulnerable to adversarial attacks, inputs with perturbations designed to intentionally trigger misclassification. Current adversarial methods directly alter pixel colors and evaluate against pixel norm-balls: pixel perturbations smaller than a specified ma…

Cited by 120Source ↗PDF ↗Scholar ↗
2019

Kernel Change-point Detection with Auxiliary Deep Generative Models

Wei-Cheng Chang, Chun-Liang Li, Yiming Yang, Barnabás Póczos
ICLR 2019poster

Detecting the emergence of abrupt property changes in time series is a challenging problem. Kernel two-sample test has been studied for this task which makes fewer assumptions on the distributions than traditional parametric approaches. However, selecting kernels is non-trivial in practice. Although…

Cited by 100Source ↗PDF ↗code ↗Scholar ↗
2019

LBS Autoencoder: Self-Supervised Fitting of Articulated Meshes to Point Clouds

Chun-Liang Li, Tomas Simon, Jason Saragih, Barnabas Poczos, Yaser Sheikh
CVPR 2019poster

We present LBS-AE; a self-supervised autoencoding algorithm for fitting articulated mesh models to point clouds. As input, we take a sequence of point clouds to be registered as well as an artist-rigged mesh, i.e. a template mesh equipped with a linear-blend skinning (LBS) deformation space paramete…

Cited by 51PDF ↗Scholar ↗
2018

Nonparametric Density Estimation under Adversarial Losses

Shashank Singh, Ananya Uppal, Boyue Li, Chun-Liang Li, Manzil Zaheer, Barnabas Poczos
NeurIPS 2018poster

We study minimax convergence rates of nonparametric density estimation under a large class of loss functions called ``adversarial losses'', which, besides classical L^p losses, includes maximum mean discrepancy (MMD), Wasserstein distance, and total variation distance. These losses are closely relat…

Cited by 94Source ↗PDF ↗Scholar ↗
2017

MMD GAN: Towards Deeper Understanding of Moment Matching Network

Chun-Liang Li, Wei-Cheng Chang, Yu Cheng, Yiming Yang, Barnabas Poczos
NeurIPS 2017poster

Generative moment matching network (GMMN) is a deep generative model that differs from Generative Adversarial Network (GAN) by replacing the discriminator in GAN with a two-sample test based on kernel maximum mean discrepancy (MMD). Although some theoretical guarantees of MMD have been studied, the…

Cited by 924Source ↗PDF ↗code ↗Scholar ↗
2017

One Network to Solve Them All -- Solving Linear Inverse Problems Using Deep Projection Models

J. H. Rick Chang, Chun-Liang Li, Barnabas Poczos, B. V. K. Vijaya Kumar, Aswin C. Sankaranarayanan
ICCV 2017oral

While deep learning methods have achieved state-of-the-art performance in many challenging inverse problems like image inpainting and super-resolution, they invariably involve problem-specific training of the networks. Under this approach, each inverse problem requires its own dedicated network. In…

Cited by 410PDF ↗code ↗Scholar ↗
2016

High Dimensional Bayesian Optimization via Restricted Projection Pursuit Models

Chun-Liang Li, Kirthevasan Kandasamy, Barnabas Poczos, Jeff Schneider
AISTATS 2016poster

Bayesian Optimization (BO) is commonly used to optimize blackbox objective functions which are expensive to evaluate. A common approach is based on using Gaussian Process (GP) to model the objective function. Applying GP to higher dimensional settings is generally difficult due to the curse of dimen…

Cited by 103Source ↗PDF ↗Scholar ↗
2016

Rivalry of Two Families of Algorithms for Memory-Restricted Streaming PCA

Chun-Liang Li, Hsuan-Tien Lin, Chi-Jen Lu
AISTATS 2016poster

We study the problem of recovering the subspace spanned by the first k principal components of d-dimensional data under the streaming setting, with a memory bound of O(kd). Two families of algorithms are known for this problem. The first family is based on the framework of stochastic gradient descen…

Cited by 40Source ↗PDF ↗Scholar ↗