AIConfPaper
← Search

Tengyu Xu

16 accepted papers

2025

Learning Auxiliary Tasks Improves Reference-Free Hallucination Detection in Open-Domain Long-Form Generation

Chengwei Qin, Wenxuan Zhou, Karthik Abinav Sankararaman, Nanshu Wang, Tengyu Xu, Alexander Radovic +7
ACL 2025short

Hallucination, the generation of factually incorrect information, remains a significant challenge for large language models (LLMs), especially in open-domain long-form generation. Existing approaches for detecting hallucination in long-form tasks either focus on limited domains or rely heavily on ex…

Cited by 0Source ↗PDF ↗Scholar ↗
2025

Think Smarter not Harder: Adaptive Reasoning with Inference Aware Optimization

Zishun Yu, Tengyu Xu, Di Jin, Karthik Abinav Sankararaman, Yun He, Wenxuan Zhou +6
ICML 2025poster

Solving mathematics problems has been an intriguing capability of large language models, and many efforts have been made to improve reasoning by extending reasoning length, such as through self-correction and extensive long chain-of-thoughts. While promising in problem-solving, advanced long reasoni…

Cited by 7Source ↗PDF ↗Scholar ↗
2022

A Unifying Framework of Off-Policy General Value Function Evaluation

Tengyu Xu, Zhuoran Yang, Zhaoran Wang, Yingbin Liang
NeurIPS 2022accept

General Value Function (GVF) is a powerful tool to represent both the {\em predictive} and {\em retrospective} knowledge in reinforcement learning (RL). In practice, often multiple interrelated GVFs need to be evaluated jointly with pre-collected off-policy samples. In the literature, the gradient t…

Cited by 3Source ↗PDF ↗Scholar ↗
2022

Deterministic policy gradient: Convergence analysis

Huaqing. Xiong, Tengyu Xu, Lin Zhao, Yingbin Liang, Wei Zhang
UAI 2022poster

The deterministic policy gradient (DPG) method proposed in Silver et al. [2014] has been demonstrated to exhibit superior performance particularly for applications with multi-dimensional and continuous action spaces. However, it remains unclear whether DPG converges, and if so, how fast it converges…

Cited by 25Source ↗PDF ↗Scholar ↗
2022

Model-Based Offline Meta-Reinforcement Learning with Regularization

Sen Lin, Jialin Wan, Tengyu Xu, Yingbin Liang, Junshan Zhang
ICLR 2022poster

Existing offline reinforcement learning (RL) methods face a few major challenges, particularly the distributional shift between the learned policy and the behavior policy. Offline Meta-RL is emerging as a promising approach to address these challenges, aiming to learn an informative meta-policy from…

Cited by 24Source ↗PDF ↗Scholar ↗
2022

PER-ETD: A Polynomially Efficient Emphatic Temporal Difference Learning Method

Ziwei Guan, Tengyu Xu, Yingbin Liang
ICLR 2022poster

Emphatic temporal difference (ETD) learning (Sutton et al., 2016) is a successful method to conduct the off-policy value function evaluation with function approximation. Although ETD has been shown to converge asymptotically to a desirable value function, it is well-known that ETD often encounters a…

Cited by 9Source ↗PDF ↗Scholar ↗
2021

CRPO: A New Approach for Safe Reinforcement Learning with Convergence Guarantee

Tengyu Xu, Yingbin Liang, Guanghui Lan
ICML 2021spotlight

In safe reinforcement learning (SRL) problems, an agent explores the environment to maximize an expected total reward and meanwhile avoids violation of certain constraints on a number of expected total costs. In general, such SRL problems have nonconvex objective functions subject to multiple noncon…

Cited by 169Source ↗PDF ↗Scholar ↗
2021

Doubly Robust Off-Policy Actor-Critic: Convergence and Optimality

Tengyu Xu, Zhuoran Yang, Zhaoran Wang, Yingbin Liang
ICML 2021spotlight

Designing off-policy reinforcement learning algorithms is typically a very challenging task, because a desirable iteration update often involves an expectation over an on-policy distribution. Prior off-policy actor-critic (AC) algorithms have introduced a new critic that uses the density ratio for a…

Cited by 38Source ↗PDF ↗Scholar ↗
2021

Non-asymptotic Convergence of Adam-type Reinforcement Learning Algorithms under Markovian Sampling

Huaqing Xiong, Tengyu Xu, Yingbin Liang, Wei Zhang
AAAI 2021technical

Despite the wide applications of Adam in reinforcement learning (RL), the theoretical convergence of Adam-type RL algorithms has not been established. This paper provides the first such convergence analysis for two fundamental RL algorithms of policy gradient (PG) and temporal difference (TD) learni…

Cited by 41Source ↗PDF ↗Scholar ↗
2021

Proximal Gradient Descent-Ascent: Variable Convergence under KŁ Geometry

Ziyi Chen, Yi Zhou, Tengyu Xu, Yingbin Liang
ICLR 2021poster

The gradient descent-ascent (GDA) algorithm has been widely applied to solve minimax optimization problems. In order to achieve convergent policy parameters for minimax optimization, it is important that GDA generates convergent variable sequences rather than convergent sequences of function value o…

Cited by 37Source ↗PDF ↗Scholar ↗
2021

Sample Complexity Bounds for Two Timescale Value-based Reinforcement Learning Algorithms

Tengyu Xu, Yingbin Liang
AISTATS 2021poster

Two timescale stochastic approximation (SA) has been widely used in value-based reinforcement learning algorithms. In the policy evaluation setting, it can model the linear and nonlinear temporal difference learning with gradient correction (TDC) algorithms as linear SA and nonlinear SA, respectivel…

Cited by 41Source ↗PDF ↗Scholar ↗
2021

When Will Generative Adversarial Imitation Learning Algorithms Attain Global Convergence

Ziwei Guan, Tengyu Xu, Yingbin Liang
AISTATS 2021poster

Generative adversarial imitation learning (GAIL) is a popular inverse reinforcement learning approach for jointly optimizing policy and reward from expert trajectories. A primary question about GAIL is whether applying a certain policy gradient algorithm to GAIL attains a global minimizer (i.e., yie…

Cited by 25Source ↗PDF ↗Scholar ↗
2019

Two Time-scale Off-Policy TD Learning: Non-asymptotic Analysis over Markovian Samples

Tengyu Xu, Shaofeng Zou, Yingbin Liang
NeurIPS 2019poster

Gradient-based temporal difference (GTD) algorithms are widely used in off-policy learning scenarios. Among them, the two time-scale TD with gradient correction (TDC) algorithm has been shown to have superior performance. In contrast to previous studies that characterized the non-asymptotic converge…

Cited by 98Source ↗PDF ↗Scholar ↗