Dzmitry Tsetserukou

FlightDiffusion: Revolutionizing Autonomous Drone Training with Diffusion Model Generating FPV Video

We present FlightDiffusion, a diffusion-based framework for training autonomous drones from first-person-view (FPV) video. The model generates FPV video sequences from a single frame and a text prompt, and derives corresponding state-action trajectories for task-conditioned navigation. FlightDiffusi…

HIPPO-MAT: Decentralized Task Allocation Using GraphSAGE and Multi-Agent Deep Reinforcement Learning

CognitiveOS: Large Multimodal Model Based System to Endow Any Type of Robot with Generative AI

This paper introduces CognitiveOS, the first operating system designed for cognitive robots capable of functioning across diverse robotic platforms. CognitiveOS is structured as a multi-agent system comprising modules built upon a transformer architecture, facilitating communication through an inter

ImpedanceGPT: VLM-driven Impedance Control of Swarm of Mini-drones for Intelligent Navigation in Dynamic Environment

Swarm robotics plays a crucial role in enabling autonomous operations in dynamic and unpredictable environments. However, a major challenge remains ensuring safe and efficient navigation in environments shared by both dynamic alive (e.g., humans) and dynamic inanimate (e.g., non-living objects) obst

Industry 6.0: New Generation of Industry driven by Generative AI and Swarm of Heterogeneous Robots

This paper presents the concept of Industry 6.0, which introduces the world’s first fully automated production system that autonomously handles the entire product design and manufacturing process based on user-provided natural language descriptions. By leveraging generative AI, the system automates

METDrive: Multimodal End-to-End Autonomous Driving with Temporal Guidance

Multimodal end-to-end autonomous driving has shown promising advancements in recent work. By embedding more modalities into end-to-end networks, the system's understanding of both static and dynamic aspects of the driving environment is enhanced, thereby improving the safety of autonomous driving. I

GrainGrasp: Dexterous Grasp Generation with Fine-grained Contact Guidance

One goal of dexterous robotic grasping is to allow robots to handle objects with the same level of flexibility and adaptability as humans. However, it remains a challenging task to generate an optimal grasping strategy for dexterous hands, especially when it comes to delicate manipulation and accura…

OmniRace: 6D Hand Pose Estimation for Intuitive Guidance of Racing Drone

This paper presents the OmniRace approach to controlling a racing drone with 6-degree of freedom (DoF) hand pose estimation and gesture recognition. To our knowledge, this is the first technology enabling low-level control of high-speed drones through gestures. OmniRace employs a gesture interface b…

Hierarchical Whole-body Control of the cable-Suspended Aerial Manipulator endowed with Winch-based Actuation

During operation, aerial manipulation systems are affected by various disturbances. Among them is a gravitational torque caused by the weight of the robotic arm. Common propeller-based actuation is ineffective against such disturbances because of possible overheating and high power consumption. To o…

Multi-Sensor Large-Scale Dataset for Multi-View 3D Reconstruction

We present a new multi-sensor dataset for multi-view 3D surface reconstruction. It includes registered RGB and depth data from sensors of different resolutions and modalities: smartphones, Intel RealSense, Microsoft Kinect, industrial cameras, and structured-light scanner. The scenes are selected to…

NFOMP: Neural Field for Optimal Motion Planner of Differential Drive Robots With Nonholonomic Constraints

Optimal motion planning is one of the most critical problems in mobile robotics. On the one hand, classical sampling-based methods propose asymptotically optimal solutions to this problem. However, these planners cannot achieve smooth and short trajectories in reasonable calculation time. On the oth

DeltaCharger: Charging Robot With Inverted Delta Mechanism and CNN-Driven High Fidelity Tactile Perception for Precise 3D Positioning

DeltaCharger is a novel charging robot with an Inverted Delta structure for 3D positioning of electrodes to achieve robust and safe transferring energy between two mobile robots. The embedded high-fidelity tactile sensors allow to estimate the angular, vertical and horizontal misalignments between e

Optimal Oscillation Damping Control of cable-Suspended Aerial Manipulator with a Single IMU Sensor

This paper presents a design of oscillation damping control for the cable-Suspended Aerial Manipulator (SAM). The SAM is modeled as a double pendulum, and it can generate a body wrench as a control action. The main challenge is the fact that there is only one onboard IMU sensor which does not provid…

TT-TSDF: Memory-Efficient TSDF with Low-Rank Tensor Train Decomposition

In this paper we apply the low-rank Tensor Train decomposition for compression and operations on 3D objects and scenes represented by volumetric distance functions. Our study shows that not only it allows for a very efficient compression of the high-resolution TSDF maps (up to three orders of magnit…

WareVision: CNN Barcode Detection-Based UAV Trajectory Optimization for Autonomous Warehouse Stocktaking

This letter presents a heterogeneous Unmanned Aerial Vehicle (UAV)-based robotic system for real-time barcode detection and scanning using Convolutional Neural Networks (CNN). The proposed approach improves the UAV's localization using scanned barcodes as landmarks in a real warehouse with low-light

Asynchronous Behavior Trees with Memory aimed at Aerial Vehicles with Redundancy in Flight Controller

Complex aircraft systems are becoming a target for automation. For successful operation, they require both efficient and readable mission execution system (MES). Flight control computer (FCC) units, as well as all important subsystems, are often duplicated. Discrete nature of MES does not allow smal…

Development of SAM: cable-Suspended Aerial Manipulator

High risk of a collision between rotor blades and the obstacles in a complex environment imposes restrictions on the aerial manipulators. To solve this issue, a novel system cable-Suspended Aerial Manipulator (SAM) is presented in this paper. Instead of attaching a robotic manipulator directly to an…

IceVisionSet: lossless video dataset collected on Russian winter roads with traffic sign annotations

Ability of autonomous vehicles to operate in complex dynamic environments requires, among other things, fast and accurate perception of surroundings, which includes recognition and tracking of traffic signs.For development and testing of modern sophisticated computer vision systems large and diverse…

AA-ICP: Iterative Closest Point with Anderson Acceleration

Iterative Closest Point (ICP) is a widely used method for performing scan-matching and registration. Being simple and robust, this method is still computationally expensive and may be challenging to use in real-time applications with limited resources on mobile platforms. In this paper we propose a…

DroneGear: A Novel Robotic Landing Gear With Embedded Optical Torque Sensors for Safe Multicopter Landing on an Uneven Surface

One of the most serious disadvantages of the vertical takeoff and landing flying robots including multicopters is the necessity of the flat stable area for safe landing. The ability to land and takeoff from uneven surfaces and moving basements would help the rotorcrafts in many important application