AIConfPaper
← Search

Aleksandra Faust

35 accepted papers

2026

Triangle Multiplication is All You Need for Biomolecular Structure Representations

Jeffrey Ouyang-Zhang, Pranav Murugan, Daniel Jesus Diaz, Gianluca Scarpellini, Richard Strong Bowen, Nate Gruver +4
ICLR 2026poster

AlphaFold has transformed protein structure prediction, but emerging applications such as virtual ligand screening, proteome-wide folding, and de novo binder design demand predictions at a massive scale, where runtime and memory costs become prohibitive. A major bottleneck lies in the Pairformer bac…

Cited by 0Source ↗code ↗Scholar ↗
2025

ElasticTok: Adaptive Tokenization for Image and Video

Wilson Yan, Volodymyr Mnih, Aleksandra Faust, Matei Zaharia, Pieter Abbeel, Hao Liu
ICLR 2025poster

Efficient video tokenization remains a key bottleneck in learning general purpose vision models that are capable of processing long video sequences. Prevailing approaches are restricted to encoding videos to a fixed number of tokens, where too few tokens will result in overly lossy encodings, and to…

Cited by 5Source ↗PDF ↗code ↗Scholar ↗
2025

Inference-Aware Fine-Tuning for Best-of-N Sampling in Large Language Models

Yinlam Chow, Guy Tennenholtz, Izzeddin Gur, Vincent Zhuang, Bo Dai, Aviral Kumar +4
ICLR 2025poster

Recent studies indicate that effectively utilizing inference-time compute is crucial for attaining good performance from large language models (LLMs). Specifically, the Best-of-N (BoN) inference strategy, where an LLM generates multiple responses and a verifier selects the best, has shown strong emp…

Cited by 18Source ↗PDF ↗Scholar ↗
2025

Training Language Models to Self-Correct via Reinforcement Learning

Aviral Kumar, Vincent Zhuang, Rishabh Agarwal, Yi Su, John D Co-Reyes, Avi Singh +12
ICLR 2025oral

Self-correction is a highly desirable capability of large language models (LLMs), yet it has consistently been found to be largely ineffective in modern LLMs. Current methods for training self-correction typically depend on either multiple models, a more advanced model, or additional forms of super…

Cited by 113Source ↗PDF ↗Scholar ↗
2024

A Real-World WebAgent with Planning, Long Context Understanding, and Program Synthesis

Izzeddin Gur, Hiroki Furuta, Austin V Huang, Mustafa Safdari, Yutaka Matsuo, Douglas Eck +1
ICLR 2024oral

Pre-trained large language models (LLMs) have recently achieved better generalization and sample efficiency in autonomous web automation. However, the performance on real-world websites has still suffered from (1) open domainness, (2) limited context length, and (3) lack of inductive bias on HTML. W…

Cited by 233Source ↗PDF ↗Scholar ↗
2024

Geometric-Averaged Preference Optimization for Soft Preference Labels

Hiroki Furuta, Kuang-Huei Lee, Shixiang Shane Gu, Yutaka Matsuo, Aleksandra Faust, Heiga Zen +1
NeurIPS 2024poster

Many algorithms for aligning LLMs with human preferences assume that human preferences are binary and deterministic. However, human preferences can vary across individuals, and therefore should be represented distributionally. In this work, we introduce the distributional soft preference labels and…

Cited by 7Source ↗PDF ↗Scholar ↗
2024

Many-Shot In-Context Learning

Rishabh Agarwal, Avi Singh, Lei M Zhang, Bernd Bohnet, Luis Rosias, Stephanie C.Y. Chan +9
NeurIPS 2024spotlight

Large language models (LLMs) excel at few-shot in-context learning (ICL) -- learning from a few examples provided in context at inference, without any weight updates. Newly expanded context windows allow us to investigate ICL with hundreds or thousands of examples – the many-shot regime. Going from…

Cited by 115Source ↗PDF ↗Scholar ↗
2024

Multimodal Web Navigation with Instruction-Finetuned Foundation Models

Hiroki Furuta, Kuang-Huei Lee, Ofir Nachum, Yutaka Matsuo, Aleksandra Faust, Shixiang Shane Gu +1
ICLR 2024poster

The progress of autonomous web navigation has been hindered by the dependence on billions of exploratory interactions via online reinforcement learning, and domain-specific model designs that make it difficult to leverage generalization from rich out-of-domain data. In this work, we study data-drive…

Cited by 111Source ↗PDF ↗Scholar ↗
2024

Position: Levels of AGI for Operationalizing Progress on the Path to AGI

Meredith Ringel Morris, Jascha Sohl-Dickstein, Noah Fiedel, Tris Warkentin, Allan Dafoe, Aleksandra Faust +2
ICML 2024spotlight

We propose a framework for classifying the capabilities and behavior of Artificial General Intelligence (AGI) models and their precursors. This framework introduces levels of AGI performance, generality, and autonomy, providing a common language to compare models, assess risks, and measure progress…

Cited by 115Source ↗PDF ↗Scholar ↗
2024

Stop Regressing: Training Value Functions via Classification for Scalable Deep RL

Jesse Farebrother, Jordi Orbay, Quan Vuong, Adrien Ali Taiga, Yevgen Chebotar, Ted Xiao +6
ICML 2024oral

Value functions are an essential component in deep reinforcement learning (RL), that are typically trained via mean squared error regression to match bootstrapped target values. However, scaling value-based RL methods to large networks has proven challenging. This difficulty is in stark contrast to…

Cited by 60Source ↗PDF ↗Scholar ↗
2023

CLUTR: Curriculum Learning via Unsupervised Task Representation Learning

Abdus Salam Azad, Izzeddin Gur, Jasper Emhoff, Nathaniel Alexis, Aleksandra Faust, Pieter Abbeel +1
ICML 2023poster

Reinforcement Learning (RL) algorithms are often known for sample inefficiency and difficult generalization. Recently, Unsupervised Environment Design (UED) emerged as a new paradigm for zero-shot generalization by simultaneously learning a task distribution and agent policies on the generated tasks…

Cited by 17Source ↗PDF ↗code ↗Scholar ↗
2023

Enhancing Value Estimation Policies by Post-Hoc Symmetry Exploitation in Motion Planning Tasks

Yazied Hasan, Ariana M. Villegas-Suarez, Evan C. Carter, Aleksandra Faust, Lydia Tapia
IROS 2023poster

Motion planning tasks are often innately invariant to certain geometric transformations, or in other words, symmetric. This property, however, is not always reflected in learned policies that are trained on these tasks. Although this asymmetry can be addressed through data augmentation or additional…

Cited by 0Source ↗Scholar ↗
2023

Imitation Is Not Enough: Robustifying Imitation with Reinforcement Learning for Challenging Driving Scenarios

Yiren Lu, Justin Fu, George Tucker, Xinlei Pan, Eli Bronstein, Rebecca Roelofs +6
IROS 2023poster

Imitation learning (IL) is a simple and powerful way to use high-quality human driving data, which can be collected at scale, to produce human-like behavior. However, policies based on imitation learning alone often fail to sufficiently account for safety and reliability concerns. In this paper, we…

Cited by 106Source ↗Scholar ↗
2023

SayTap: Language to Quadrupedal Locomotion

Yujin Tang, Wenhao Yu, Jie Tan, Heiga Zen, Aleksandra Faust, Tatsuya Harada
CoRL 2023poster

Large language models (LLMs) have demonstrated the potential to perform high-level planning. Yet, it remains a challenge for LLMs to comprehend low-level commands, such as joint angle targets or motor torques. This paper proposes an approach to use foot contact patterns as an interface that bridges…

Cited by 45Source ↗code ↗Scholar ↗
2023

Understanding HTML with Large Language Models

Izzeddin Gur, Ofir Nachum, Yingjie Miao, Mustafa Safdari, Austin V Huang, Aakanksha Chowdhery +3
EMNLP 2023long findings

Large language models (LLMs) have shown exceptional performance on a variety of natural language tasks. Yet, their capabilities for HTML understanding – i.e., parsing the raw HTML of a webpage, with applications to automation of web-based tasks, crawling, and browser-…

Cited by 0Source ↗Scholar ↗
2023

Waymax: An Accelerated, Data-Driven Simulator for Large-Scale Autonomous Driving Research

Cole Gulino, Justin Fu, Wenjie Luo, George Tucker, Eli Bronstein, Yiren Lu +16
NeurIPS 2023poster

Simulation is an essential tool to develop and benchmark autonomous vehicle planning software in a safe and cost-effective manner. However, realistic simulation requires accurate modeling of multi-agent interactive behaviors to be trustworthy, behaviors which can be highly nuanced and complex. To ad…

Cited by 116Source ↗PDF ↗Scholar ↗
2022

Fast inference and transfer of compositional task structures for few-shot task generalization

Sungryull Sohn, Hyunjae Woo, Jongwook Choi, Lyubing Qiang, Izzeddin Gur, Aleksandra Faust +1
UAI 2022poster

We tackle real-world problems with complex structures beyond the pixel-based game or simulator. We formulate it as a few-shot reinforcement learning problem where a task is characterized by a subtask graph that defines a set of subtasks and their dependencies that are unknown to the agent. Different…

Cited by 0Source ↗PDF ↗Scholar ↗
2022

Less Is More: Generating Grounded Navigation Instructions From Landmarks

Su Wang, Ceslee Montgomery, Jordi Orbay, Vighnesh Birodkar, Aleksandra Faust, Izzeddin Gur +4
CVPR 2022poster

We study the automatic generation of navigation instructions from 360-degree images captured on indoor routes. Existing generators suffer from poor visual grounding, causing them to rely on language priors and hallucinate objects. Our MARKY-MT5 system addresses this by focusing on visual landmarks;…

Cited by 61PDF ↗code ↗Scholar ↗
2022

Multi-Task Learning with Sequence-Conditioned Transporter Networks

Michael H. Lim, Andy Zeng, Brian Ichter, Maryam Bandari, Erwin Coumans, Claire Tomlin +2
ICRA 2022poster

Enabling robots to solve multiple manipulation tasks has a wide range of industrial applications. While learning-based approaches enjoy flexibility and generalizability, scaling these approaches to solve such compositional tasks remains a challenge. In this work, we aim to solve multi-task learning…

Cited by 15Source ↗Scholar ↗
2021

Avoidance Critical Probabilistic Roadmaps for Motion Planning in Dynamic Environments

Felipe Felix Arias, Brian Ichter, Aleksandra Faust, Nancy M. Amato
ICRA 2021poster

Motion planning among dynamic obstacles is an essential capability towards navigation in the real-world. Sampling-based motion planning algorithms find solutions by approximating the robot’s configuration space through a graph representation, predicting or computing obstacles’ trajectories, and find…

Cited by 22Source ↗Scholar ↗
2021

Environment Generation for Zero-Shot Compositional Reinforcement Learning

Izzeddin Gur, Natasha Jaques, Yingjie Miao, Jongwook Choi, Manoj Tiwari, Honglak Lee +1
NeurIPS 2021poster

Many real-world problems are compositional – solving them requires completing interdependent sub-tasks, either in series or in parallel, that can be represented as a dependency graph. Deep reinforcement learning (RL) agents often struggle to learn such complex tasks due to the long time horizons and…

Cited by 50Source ↗PDF ↗code ↗Scholar ↗
2021

Evolving Reinforcement Learning Algorithms

John D Co-Reyes, Yingjie Miao, Daiyi Peng, Esteban Real, Quoc V Le, Sergey Levine +2
ICLR 2021oral

We propose a method for meta-learning reinforcement learning algorithms by searching over the space of computational graphs which compute the loss function for a value-based model-free RL agent to optimize. The learned algorithms are domain-agnostic and can generalize to new environments not seen du…

Cited by 99Source ↗PDF ↗code ↗Scholar ↗
2021

Tiny Robot Learning (tinyRL) for Source Seeking on a Nano Quadcopter

Bardienus P. Duisterhof, Srivatsan Krishnan, Jonathan J. Cruz, Colby R. Banbury, William Fu, Aleksandra Faust +2
ICRA 2021poster

We present fully autonomous source seeking onboard a highly constrained nano quadcopter, by contributing application-specific system and observation feature design to enable inference of a deep-RL policy onboard a nano quadcopter. Our deep-RL algorithm finds a high-performance solution to a challeng…

Cited by 30Source ↗Scholar ↗
2021

Visual Navigation Among Humans With Optimal Control as a Supervisor

Varun Tolani, Somil Bansal, Aleksandra Faust, Claire J. Tomlin
RA-L 2021

Real world visual navigation requires robots to operate in unfamiliar, human-occupied dynamic environments. Navigation around humans is especially difficult because it requires anticipating their future motion, which can be quite challenging. We propose an approach that combines learning-based perce

Cited by 46Source ↗Scholar ↗
2020

Learned Critical Probabilistic Roadmaps for Robotic Motion Planning

Brian Ichter, Edward Schmerling, Tsang-Wei Edward Lee, Aleksandra Faust
ICRA 2020poster

Sampling-based motion planning techniques have emerged as an efficient algorithmic paradigm for solving complex motion planning problems. These approaches use a set of probing samples to construct an implicit graph representation of the robot's state space, allowing arbitrarily accurate representati…

Cited by 107Source ↗Scholar ↗
2020

Model-based Reinforcement Learning for Decentralized Multiagent Rendezvous

Rose Wang, J. Chase Kew, Dennis Lee, Tsang-Wei Lee, Tingnan Zhang, Brian Ichter +2
CoRL 2020

Collaboration requires agents to align their goals on the fly. Underlying the human ability to align goals with other agents is their ability to predict the intentions of others and actively update their own plans. We propose hierarchical predictive planning (HPP), a model-based reinforcement learni

Cited by 0Source ↗PDF ↗Scholar ↗
2020

Safe Policy Learning for Continuous Control

Yinlam Chow, Ofir Nachum, Aleksandra Faust, Edgar Dueñez-Guzman, Mohammad Ghavamzadeh
CoRL 2020

We study continuous action reinforcement learning problems in which it is crucial that the agent interacts with the environment only through near-safe policies, i.e., policies that keep the agent in desirable situations, both during training and at convergence. We formulate these problems as {\em co

Cited by 0Source ↗PDF ↗Scholar ↗
2020

Zero-shot Imitation Learning from Demonstrations for Legged Robot Visual Navigation

Xinlei Pan, Tingnan Zhang, Brian Ichter, Aleksandra Faust, Jie Tan, Sehoon Ha
ICRA 2020poster

Imitation learning is a popular approach for training effective visual navigation policies. However, collecting expert demonstrations for legged robots is challenging as these robots can be hard to control, move slowly, and cannot operate continuously for long periods of time. In this work, we propo…

Cited by 32Source ↗Scholar ↗
2019

Comparison of Deep Reinforcement Learning Policies to Formal Methods for Moving Obstacle Avoidance

Arpit Garg, Hao-Tien Lewis Chiang, Satomi Sugaya, Aleksandra Faust, Lydia Tapia
IROS 2019poster

Deep Reinforcement Learning (RL) has recently emerged as a solution for moving obstacle avoidance. Deep RL learns to simultaneously predict obstacle motions and corresponding avoidance actions directly from robot sensors, even for obstacles with different dynamics models. However, deep RL methods ty…

Cited by 14Source ↗Scholar ↗
2019

RL-RRT: Kinodynamic Motion Planning via Learning Reachability Estimators From RL Policies

Hao-Tien Lewis Chiang, Jasmine Hsu, Marek Fiser, Lydia Tapia, Aleksandra Faust
RA-L 2019

This letter addresses two challenges facing samplingbased kinodynamic motion planning: a way to identify good candidate states for local transitions and the subsequent computationally intractable steering between these candidate states. Through the combination of sampling-based planning, a Rapidly E

Cited by 157Source ↗Scholar ↗
2018

PRM-RL: Long-range Robotic Navigation Tasks by Combining Reinforcement Learning and Sampling-Based Planning

Aleksandra Faust, Kenneth Oslund, Oscar Ramirez, Anthony Francis, Lydia Tapia, Marek Fiser +1
ICRA 2018poster

We present PRM-RL, a hierarchical method for long-range navigation task completion that combines sampling-based path planning with reinforcement learning (RL). The RL agents learn short-range, point-to-point navigation policies that capture robot dynamics and task constraints without knowledge of th…

Cited by 401Source ↗Scholar ↗
2016

Avoiding moving obstacles with stochastic hybrid dynamics using PEARL: PrEference Appraisal Reinforcement Learning

Aleksandra Faust, Hao-Tien Chiang, Nathanael Rackley, Lydia Tapia
ICRA 2016

Manual derivation of optimal robot motions for task completion is difficult, especially when a robot is required to balance its actions between opposing preferences. One solution has been proposed to automatically learn near optimal motions with Reinforcement Learning (RL). This has been successful

Cited by 19Source ↗Scholar ↗