← Search

Samuele Tosatto

10 accepted papers

2026

Dynamic Sparsity: Challenging Common Sparsity Assumptions for Learning World Models in Robotic Reinforcement Learning Benchmarks

AAAI 2026technical

The use of learned dynamics models, also known as world models, can improve the sample efficiency of reinforcement learning. Recent work suggests that the underlying causal graphs of such dynamics models are sparsely connected, with each of the future state variables depending only on a small subset

Cited by 0SourcePDFScholar
2024

Local Linearity is All You Need (in Data-Driven Teleoperation)

IROS 2024poster

One of the critical aspects of assistive robotics is to provide a control system of a high-dimensional robot from a low-dimensional user input (i.e. a 2D joystick). Data-driven teleoperation seeks to provide an intuitive user interface called an action map to map the low dimensional input to robot v…

Cited by 0SourceScholar
2023

Deep Probabilistic Movement Primitives with a Bayesian Aggregator

IROS 2023poster

Movement primitives are trainable parametric models that reproduce robotic movements starting from a limited set of demonstrations. Previous works proposed simple linear models that exhibited high sample efficiency and generalization power by allowing temporal modulation of move-ments (reproducing m…

Cited by 5SourceScholar
2023

Dynamic Decision Frequency with Continuous Options

IROS 2023poster

In classic reinforcement learning algorithms, agents make decisions at discrete and fixed time intervals. The duration between decisions becomes a crucial hyperparameter, as setting it too short may increase the problem's difficulty by requiring the agent to make numerous decisions to achieve its go…

Cited by 9SourcecodeScholar
2022

A Temporal-Difference Approach to Policy Gradient Estimation

ICML 2022spotlight

The policy gradient theorem (Sutton et al., 2000) prescribes the usage of a cumulative discounted state distribution under the target policy to approximate the gradient. Most algorithms based on this theorem, in practice, break this assumption, introducing a distribution shift that can cause the con…

2022

An Alternate Policy Gradient Estimator for Softmax Policies

AISTATS 2022poster

Policy gradient (PG) estimators are ineffective in dealing with softmax policies that are sub-optimally saturated, which refers to the situation when the policy concentrates its probability mass on sub-optimal actions. Sub-optimal policy saturation may arise from bad policy initialization or sudden…

2022

Model-free Policy Learning with Reward Gradients

AISTATS 2022poster

Despite the increasing popularity of policy gradient methods, they are yet to be widely utilized in sample-scarce applications, such as robotics. The sample efficiency could be improved by making best usage of available information. As a key component in reinforcement learning, the reward function i…

2021

Contextual Latent-Movements Off-Policy Optimization for Robotic Manipulation Skills

ICRA 2021poster

Parameterized movement primitives have been extensively used for imitation learning of robotic tasks. However, the high-dimensionality of the parameter space hinders the improvement of such primitives in the reinforcement learning (RL) setting, especially for learning with physical robots. In this p…

Cited by 23SourcecodeScholar
2020

A Nonparametric Off-Policy Policy Gradient

AISTATS 2020poster

Reinforcement learning (RL) algorithms still suffer from high sample complexity despite outstanding recent successes. The need for intensive interactions with the environment is especially observed in many widely popular policy gradient algorithms that perform updates using on-policy samples. The pr…