AIConfPaper
← Search

Chenyang Tao

22 accepted papers

2025

Do Not Design, Learn: A Trainable Scoring Function for Uncertainty Estimation in Generative LLMs

Duygu Nur Yaldiz, Yavuz Faruk Bakman, Baturalp Buyukates, Chenyang Tao, Anil Ramakrishna, Dimitrios Dimitriadis +2
NAACL 2025findings

Uncertainty estimation (UE) of generative large language models (LLMs) is crucial for evaluating the reliability of generated sequences. A significant subset of UE methods utilize token probabilities to assess uncertainty, aggregating multiple token probabilities into a single UE score using a scori…

Cited by 1Source ↗PDF ↗Scholar ↗
2025

FLAMES: Improving LLM Math Reasoning via a Fine-Grained Analysis of the Data Synthesis Pipeline

Parker Seegmiller, Kartik Mehta, Soumya Saha, Chenyang Tao, Shereen Oraby, Arpit Gupta +3
EMNLP 2025

Recent works improving LLM math reasoning with synthetic data have used unique setups, making comparison of data synthesis strategies impractical. This leaves many unanswered questions about the roles of different factors in the synthetic data pipeline, such as the impact of filtering low-quality pr

Cited by 0Source ↗PDF ↗code ↗Scholar ↗
2024

CroMo-Mixup: Augmenting Cross-Model Representations for Continual Self-Supervised Learning

Erum Mushtaq*, Duygu Nur Yaldiz, Yavuz Faruk Bakman, Jie Ding, Chenyang Tao, Dimitrios Dimitriadis +1
ECCV 2024poster

"Continual self-supervised learning (CSSL) learns a series of tasks sequentially on the unlabeled data. Two main challenges of continual learning are catastrophic forgetting and task confusion. While CSSL problem has been studied to address the catastrophic forgetting challenge, little work has been…

Cited by 2Source ↗PDF ↗code ↗Scholar ↗
2024

DiNADO: Norm-Disentangled Neurally-Decomposed Oracles for Controlling Language Models

Sidi Lu, Wenbo Zhao, Chenyang Tao, Arpit Gupta, Shanchan Wu, Tagyoung Chung +1
ICML 2024poster

NeurAlly-Decomposed Oracle (NADO) is a powerful approach for controllable generation with large language models. It is designed to avoid catastrophic forgetting while achieving guaranteed convergence to an entropy-maximized closed-form optimal solution with reasonable modeling capacity. Despite the…

Cited by 1Source ↗PDF ↗code ↗Scholar ↗
2024

MARS: Meaning-Aware Response Scoring for Uncertainty Estimation in Generative LLMs

Yavuz Faruk Bakman, Duygu Nur Yaldiz, Baturalp Buyukates, Chenyang Tao, Dimitrios Dimitriadis, Salman Avestimehr
ACL 2024long

Generative Large Language Models (LLMs) are widely utilized for their excellence in various tasks. However, their tendency to produce inaccurate or misleading outputs poses a potential risk, particularly in high-stakes environments. Therefore, estimating the correctness of generative LLM outputs is…

Cited by 13Source ↗PDF ↗code ↗Scholar ↗
2023

Unsupervised Melody-to-Lyrics Generation

Yufei Tian, Anjali Narayan-Chen, Shereen Oraby, Alessandra Cervone, Gunnar Sigurdsson, Chenyang Tao +5
ACL 2023long

Automatic melody-to-lyric generation is a task in which song lyrics are generated to go with a given melody. It is of significant practical interest and more challenging than unconstrained lyric generation as the music imposes additional constraints onto the lyrics. The training data is limited as m…

Cited by 17Source ↗PDF ↗code ↗Scholar ↗
2022

Gradient Importance Learning for Incomplete Observations

Qitong Gao, Dong Wang, Joshua David Amason, Siyang Yuan, Chenyang Tao, Ricardo Henao +3
ICLR 2022poster

Though recent works have developed methods that can generate estimates (or imputations) of the missing entries in a dataset to facilitate downstream analysis, most depend on assumptions that may not align with real-world applications and could suffer from poor performance in subsequent tasks such as…

Cited by 11Source ↗PDF ↗code ↗Scholar ↗
2022

Tight Mutual Information Estimation With Contrastive Fenchel-Legendre Optimization

Qing Guo, Junya Chen, Dong Wang, Yuewei Yang, Xinwei Deng, Jing Huang +3
NeurIPS 2022accept

Successful applications of InfoNCE (Information Noise-Contrastive Estimation) and its variants have popularized the use of contrastive variational mutual information (MI) estimators in machine learning . While featuring superior stability, these estimators crucially depend on costly large-batch trai…

Cited by 18Source ↗PDF ↗code ↗Scholar ↗
2021

APo-VAE: Text Generation in Hyperbolic Space

Shuyang Dai, Zhe Gan, Yu Cheng, Chenyang Tao, Lawrence Carin, Jingjing Liu
NAACL 2021long

Natural language often exhibits inherent hierarchical structure ingrained with complex syntax and semantics. However, most state-of-the-art deep generative models learn embeddings only in Euclidean vector space, without accounting for this structural property of language. In this paper, we investiga…

Cited by 38Source ↗PDF ↗Scholar ↗
2021

Counterfactual Representation Learning with Balancing Weights

Serge Assaad, Shuxi Zeng, Chenyang Tao, Shounak Datta, Nikhil Mehta, Ricardo Henao +2
AISTATS 2021poster

A key to causal inference with observational data is achieving balance in predictive features associated with each treatment type. Recent literature has explored representation learning to achieve this goal. In this work, we discuss the pitfalls of these strategies – such as a steep trade-off betwee…

Cited by 89Source ↗PDF ↗Scholar ↗
2021

Multi-Grained Knowledge Distillation for Named Entity Recognition

Xuan Zhou, Xiao Zhang, Chenyang Tao, Junya Chen, Bing Xu, Wei Wang +1
NAACL 2021long

Although pre-trained big models (e.g., BERT, ERNIE, XLNet, GPT3 etc.) have delivered top performance in Seq2seq modeling, their deployments in real-world applications are often hindered by the excessive computations and memory demand involved. For many applications, including named entity recognitio…

Cited by 17Source ↗PDF ↗Scholar ↗
2021

Supercharging Imbalanced Data Learning With Energy-based Contrastive Representation Transfer

Junya Chen, Zidi Xiu, Benjamin Goldstein, Ricardo Henao, Lawrence Carin, Chenyang Tao
NeurIPS 2021spotlight

Dealing with severe class imbalance poses a major challenge for many real-world applications, especially when the accurate classification and generalization of minority classes are of primary interest. In computer vision and NLP, learning from datasets with long-tail behavior is a recurring theme, e…

Cited by 14Source ↗PDF ↗code ↗Scholar ↗
2021

Variational Disentanglement for Rare Event Modeling

Zidi Xiu, Chenyang Tao, Michael Gao, Connor Davis, Benjamin A. Goldstein, Ricardo Henao
AAAI 2021technical

Combining the increasing availability and abundance of healthcare data and the current advances in machine learning methods have created renewed opportunities to improve clinical decision support systems. However, in healthcare risk prediction applications, the proportion of cases with the condition…

Cited by 14Source ↗PDF ↗code ↗Scholar ↗
2020

Reconsidering Generative Objectives For Counterfactual Reasoning

Danni Lu, Chenyang Tao, Junya Chen, Fan Li, Feng Guo, Lawrence Carin
NeurIPS 2020poster

There has been recent interest in exploring generative goals for counterfactual reasoning, such as individualized treatment effect (ITE) estimation. However, existing solutions often fail to address issues that are unique to causal inference, such as covariate balancing and (infeasible) counterfactu…

Cited by 33Source ↗PDF ↗code ↗Scholar ↗
2019

Improving Sequence-to-Sequence Learning via Optimal Transport

Liqun Chen, Yizhe Zhang, Ruiyi Zhang, Chenyang Tao, Zhe Gan, Haichao Zhang +4
ICLR 2019poster

Sequence-to-sequence models are commonly trained via maximum likelihood estimation (MLE). However, standard MLE training considers a word-level objective, predicting the next word given the previous ground-truth partial sentence. This procedure focuses on modeling local syntactic patterns, and may f…

Cited by 110Source ↗PDF ↗Scholar ↗
2019

Improving Textual Network Learning with Variational Homophilic Embeddings

Wenlin Wang, Chenyang Tao, Zhe Gan, Guoyin Wang, Liqun Chen, Xinyuan Zhang +4
NeurIPS 2019poster

The performance of many network learning applications crucially hinges on the success of network embedding algorithms, which aim to encode rich network information into low-dimensional vertex-based vector representations. This paper considers a novel variational formulation of network embeddings, wi…

Cited by 18Source ↗PDF ↗code ↗Scholar ↗
2019

Variational Annealing of GANs: A Langevin Perspective

Chenyang Tao, Shuyang Dai, Liqun Chen, Ke Bai, Junya Chen, Chang Liu +3
ICML 2019oral

The generative adversarial network (GAN) has received considerable attention recently as a model for data synthesis, without an explicit specification of a likelihood function. There has been commensurate interest in leveraging likelihood estimates to improve GAN training. To enrich the understandin…

Cited by 23Source ↗PDF ↗Scholar ↗
2018

Adversarial Text Generation via Feature-Mover's Distance

Liqun Chen, Shuyang Dai, Chenyang Tao, Haichao Zhang, Zhe Gan, Dinghan Shen +4
NeurIPS 2018poster

Generative adversarial networks (GANs) have achieved significant success in generating real-valued data. However, the discrete nature of text hinders the application of GAN to text-generation tasks. Instead of using the standard GAN objective, we propose to improve text-generation GAN via a novel ap…

Cited by 173Source ↗PDF ↗code ↗Scholar ↗
2018

Adversarial Time-to-Event Modeling

Paidamoyo Chapfuwa, Chenyang Tao, Chunyuan Li, Courtney Page, Benjamin Goldstein, Lawrence Carin Duke +1
ICML 2018oral

Modern health data science applications leverage abundant molecular and electronic health data, providing opportunities for machine learning to build statistical models to support clinical practice. Time-to-event analysis, also called survival analysis, stands as one of the most representative examp…

Cited by 130Source ↗PDF ↗code ↗Scholar ↗
2018

Chi-square Generative Adversarial Network

Chenyang Tao, Liqun Chen, Ricardo Henao, Jianfeng Feng, Lawrence Carin Duke
ICML 2018oral

To assess the difference between real and synthetic data, Generative Adversarial Networks (GANs) are trained using a distribution discrepancy measure. Three widely employed measures are information-theoretic divergences, integral probability metrics, and Hilbert space discrepancy metrics. We elucida…

Cited by 48Source ↗PDF ↗code ↗Scholar ↗
2018

Variational Inference and Model Selection with Generalized Evidence Bounds

Liqun Chen, Chenyang Tao, Ruiyi Zhang, Ricardo Henao, Lawrence Carin Duke
ICML 2018oral

Recent advances on the scalability and flexibility of variational inference have made it successful at unravelling hidden patterns in complex data. In this work we propose a new variational bound formulation, yielding an estimator that extends beyond the conventional variational bound. It naturally…

Cited by 36Source ↗PDF ↗code ↗Scholar ↗