← Search

Xiu Li

110 accepted papers

2026

$E^2$PO: Embedding-perturbed Exploration Preference Optimization for Flow Models

ICML 2026poster

Recent advancements have established Reinforcement Learning (RL) as a pivotal paradigm for aligning generative models with human intent. However, group-based optimization frameworks (e.g., GRPO) face a critical limitation: *the rapid decay of intra-group variance*. As the distinctiveness among sampl…

Cited by 0SourceScholar
2026

Align-Then-stEer: Adapting the Vision-Language Action Models through Unified Latent Guidance

ICLR 2026poster

Vision-Language-Action (VLA) models pre-trained on large, diverse datasets show remarkable potential for general-purpose robotic manipulation. However, a primary bottleneck remains in adapting these models to downstream tasks, especially when the robot's embodiment or the task itself differs from th…

Cited by 0SourcecodeScholar
2026

Attention as a Compass: Efficient Exploration for Process-Supervised RL in Reasoning Models

ICLR 2026poster

Reinforcement Learning (RL) has shown remarkable success in enhancing the reasoning capabilities of Large Language Models (LLMs). Process-Supervised RL (PSRL) has emerged as a more effective paradigm compared to outcome-based RL. However, existing PSRL approaches suffer from limited exploration effi…

Cited by 0SourcecodeScholar
2026

DiverseGRPO: Mitigating Mode Collapse in Image Generation via Diversity-Aware GRPO

CVPR 2026

Reinforcement learning (RL), particularly GRPO, improves image generation quality significantly by comparing the relative performance of images generated within the same group. However, in the later stages of training, the model tends to produce homogenized outputs, lacking creativity and visual div

Cited by 0SourceScholar
2026

Dual-Robust Cross-Domain Offline Reinforcement Learning Against Dynamics Shifts

ICLR 2026poster

Single-domain offline reinforcement learning (RL) often suffers from limited data coverage, while cross-domain offline RL handles this issue by leveraging additional data from other domains with dynamics shifts. However, existing studies primarily focus on train-time robustness (handling dynamics sh…

Cited by 0SourceScholar
2026

GenPRM: Scaling Test-Time Compute of Process Reward Models via Generative Reasoning

AAAI 2026technical

Recent advancements in Large Language Models (LLMs) have shown that it is promising to utilize Process Reward Models (PRMs) as verifiers to enhance the performance of LLMs. However, current PRMs face three key challenges: (1) limited process supervision and generalization capabilities, (2) dependenc

Cited by 0SourcePDFScholar
2026

Photon: Speedup Volume Understanding with Efficient Multimodal Large Language Models

ICLR 2026poster

Multimodal large language models are promising for clinical visual question answering tasks, but scaling to 3D imaging is hindered by high computational costs. Prior methods often rely on 2D slices or fixed-length token compression, disrupting volumetric continuity and obscuring subtle findings. We…

Cited by 0SourcecodeScholar
2026

PreciseCache: Precise Feature Caching for Efficient and High-fidelity Video Generation

ICLR 2026poster

High computational costs and slow inference hinder the practical application of video generation models. While prior works accelerate the generation process through feature caching, they often suffer from notable quality degradation. In this work, we reveal that this issue arises from their inabilit…

Cited by 0SourcecodeScholar
2026

S$^2$-Guidance: Stochastic Self-Guidance for Training-Free Enhancement of Diffusion Models

ICLR 2026poster

Classifier-free Guidance (CFG) is a widely used technique in modern diffusion models for generating high-quality samples. However, through an empirical analysis on both Gaussian mixture models with closed-form solutions and real-world data distributions, we observe a discrepancy between the suboptim…

Cited by 0SourcecodeScholar
2026

SpeakerVid-5M: A Large-Scale High-Quality Dataset for Audio-Visual Dyadic Interactive Human Generation

ICLR 2026poster

The rapid development of large-scale models has catalyzed significant breakthroughs in the digital human domain. These advanced methodologies offer high-fidelity solutions for avatar driving and rendering, leading academia to focus on the next major challenge: audio-visual dyadic interactive virtual…

Cited by 0SourcecodeScholar
2026

Taming Preference Mode Collapse via Directional Decoupling Alignment in Diffusion Reinforcement Learning

CVPR 2026

Recent studies have demonstrated significant progress in aligning text-to-image diffusion models with human preference via Reinforcement Learning from Human Feedback. However, while existing methods achieve high scores on automated reward metrics, they often lead to Preference Mode Collapse (PMC)-a

Cited by 0SourceScholar
2026

TopoMesh: High-Fidelity Mesh Autoencoding via Topological Unification

CVPR 2026

The dominant paradigm for high-fidelity 3D generation relies on a VAE-Diffusion pipeline, where the VAE's reconstruction capability sets a firm upper bound on generation quality. A fundamental challenge limiting existing VAEs is the representation mismatch between ground-truth meshes and network pre

Cited by 0SourceScholar
2026

Unsupervised Camouflaged Object Detection with Dual-Eigenvector Spectral Pseudo-Labeling and Contrastive Refinement

ICML 2026poster

Unsupervised Camouflaged Object Detection (UCOD) aims to identify objects concealed in their surroundings without relying on pixel-level labels. Existing methods rely solely on simple post-processing of DINO high-dimensional features to generate pseudo labels for training. However, these methods suf…

Cited by 0SourceScholar
2026

ViLearn: Accelerating Training Convergence of Image-to-3D Generation via Visibility Learning

CVPR 2026

Single-image-to-3D shape generation has seen remarkable progress, driven by latent diffusion models trained on the compressed latent space of 3D VAEs. However, the task remains intrinsically ill-posed: recovering complete 3D geometry--especially occluded surfaces--from a single view is inherently am

Cited by 0SourceScholar
2026

Zo3T: Zero-Shot 3D-Aware Trajectory-Guided Image-to-Video Generation via Test-Time Training

AAAI 2026technical

Trajectory-Guided image-to-video (I2V) generation aims to synthesize videos that adhere to user-specified motion instructions. Existing methods typically rely on computationally expensive fine-tuning on scarce annotated datasets. Although some zero-shot methods attempt to trajectory control in the l

Cited by 0SourcePDFScholar
2025

A Plug-and-Play Physical Motion Restoration Approach for In-the-Wild High-Difficulty Motions

ICCV 2025poster

Extracting physically plausible 3D human motion from videos is a critical task. Although existing simulation-based motion imitation methods can enhance the physical quality of daily motions estimated from monocular video capture, extending this capability to high-difficulty motions remains an open c…

2025

ADG: Ambient Diffusion-Guided Dataset Recovery for Corruption-Robust Offline Reinforcement Learning

NeurIPS 2025poster

Real-world datasets collected from sensors or human inputs are prone to noise and errors, posing significant challenges for applying offline reinforcement learning (RL). While existing methods have made progress in addressing corrupted actions and rewards, they remain insufficient for handling corru…

Cited by 0SourceScholar
2025

ART: Anonymous Region Transformer for Variable Multi-Layer Transparent Image Generation

CVPR 2025poster

Multi-layer image generation is a fundamental task that enables users to isolate, select, and edit specific image layers, thereby revolutionizing interactions with generative models. In this paper, we introduce the Anonymous Region Transformer (ART), which facilitates the direct generation of variab…

Cited by 4SourcePDFScholar
2025

AToM: Aligning Text-to-Motion Model at Event-Level with GPT-4Vision Reward

CVPR 2025poster

Recently, text-to-motion models open new possibilities for creating realistic human motion with greater efficiency and flexibility. However, aligning motion generation with event-level textual descriptions presents unique challenges due to the complex, nuanced relationship between textual prompts an…

Cited by 1SourcePDFScholar
2025

Audio-visual Controlled Video Diffusion with Masked Selective State Spaces Modeling for Natural Talking Head Generation

ICCV 2025poster

Talking head synthesis is vital for virtual avatars and human-computer interaction. However, most existing methods are typically limited to accepting control from a single primary modality, restricting their practical utility. To this end, we introduce ACTalker, an end-to-end video diffusion framewo…

2025

Cross-Domain Offline Policy Adaptation with Optimal Transport and Dataset Constraint

ICLR 2025poster

We explore cross-domain offline reinforcement learning (RL) where offline datasets from another domain can be accessed to facilitate policy learning. However, the underlying environments of the two datasets may have dynamics mismatches, incurring inferior performance when simply merging the data of…

Cited by 1SourcePDFScholar
2025

Densely Connected Parameter-Efficient Tuning for Referring Image Segmentation

AAAI 2025technical

In the domain of computer vision, Parameter-Efficient Tuning (PET) is increasingly replacing the traditional paradigm of pre-training followed by full fine-tuning. PET is particularly favored for its effectiveness in large foundation models, as it streamlines transfer learning costs and optimizes ha…

2025

Dora: Sampling and Benchmarking for 3D Shape Variational Auto-Encoders

CVPR 2025poster

Recent 3D content generation pipelines commonly employ Variational Autoencoders (VAEs) to encode shapes into compact latent representations for diffusion-based generation. However, the widely adopted uniform point sampling strategy in Shape VAE training often leads to a significant loss of geometric…

2025

GIVEPose: Gradual Intra-class Variation Elimination for RGB-based Category-Level Object Pose Estimation

CVPR 2025poster

Recent advances in RGBD-based category-level object pose estimation have been limited by their reliance on precise depth information, restricting their broader applicability. In response, RGB-based methods have been developed. Among these methods, geometry-guided pose regression that originated from…

2025

HunyuanPortrait: Implicit Condition Control for Enhanced Portrait Animation

CVPR 2025poster

We introduce HunyuanPortrait, a diffusion-based condition control method that employs implicit representations for highly controllable and lifelike portrait animation. Given a single portrait image as an appearance reference and video clips as driving templates, HunyuanPortrait can animate the chara…

2025

InstantSwap: Fast Customized Concept Swapping across Sharp Shape Differences

ICLR 2025poster

Recent advances in Customized Concept Swapping (CCS) enable a text-to-image model to swap a concept in the source image with a customized target concept. However, the existing methods still face the challenges of $\textit{\textbf{inconsistency}}$ and $\textit{\textbf{inefficiency}}$. They struggle t…

2025

Linear Differential Vision Transformer: Learning Visual Contrasts via Pairwise Differentials

NeurIPS 2025poster

Vision Transformers (ViTs) have become a universal backbone for both image recognition and image generation. Yet their Multi–Head Self–Attention (MHSA) layer still performs a quadratic query–key interaction for \emph{every} token pair, spending the bulk of computation on visually weak or redundant…

Cited by 0SourcecodeScholar
2025

LoRA-Gen: Specializing Large Language Model via Online LoRA Generation

ICML 2025poster

Recent advances have highlighted the benefits of scaling language models to enhance performance across a wide range of NLP tasks. However, these approaches still face limitations in effectiveness and efficiency when applied to domain-specific tasks, particularly for small edge-side models. We propos…

Cited by 0SourcePDFScholar
2025

MVPortrait: Text-Guided Motion and Emotion Control for Multi-view Vivid Portrait Animation

CVPR 2025poster

Recent portrait animation methods have made significant strides in generating realistic lip synchronization. However, they often lack explicit control over head movements and facial expressions, and cannot produce videos from multiple viewpoints, resulting in less controllable and expressive animati…

Cited by 4SourcePDFScholar
2025

MagicArticulate: Make Your 3D Models Articulation-Ready

CVPR 2025poster

With the explosive growth of 3D content creation, there is an increasing demand for automatically converting static 3D models into articulation-ready versions that support realistic animation. Traditional approaches rely heavily on manual annotation, which is both time-consuming and labor-intensive.…

2025

MindOmni: Unleashing Reasoning Generation in Vision Language Models with RGPO

NeurIPS 2025poster

Recent text-to-image systems face limitations in handling multimodal inputs and complex reasoning tasks. We introduce MindOmni, a unified multimodal large language model that addresses these challenges by incorporating reasoning generation through reinforcement learning. MindOmni leverages a three-p…

Cited by 0SourcecodeScholar
2025

MultiBooth: Towards Generating All Your Concepts in an Image from Text

AAAI 2025technical

This paper introduces MultiBooth, a method that generates images from texts containing various concepts from users.Despite diffusion models bringing significant advancements for customized text-to-image generation, existing methods often struggle with multi-concept scenarios due to low concept fidel…

2025

REPARO: Compositional 3D Assets Generation with Differentiable 3D Layout Alignment

ICCV 2025poster

Traditional image-to-3D models often struggle with scenes containing multipleobjects due to biases and occlusion complexities. To address this challenge, wepresent REPARO, a novel approach for compositional 3D asset generation fromsingle images. REPARO employs a two-step process: first, it extracts…

2025

Reti-Diff: Illumination Degradation Image Restoration with Retinex-based Latent Diffusion Model

ICLR 2025spotlight

Illumination degradation image restoration (IDIR) techniques aim to improve the visibility of degraded images and mitigate the adverse effects of deteriorated illumination. Among these algorithms, diffusion-based models (DM) have shown promising performance but are often burdened by heavy computatio…

2025

Revisiting Multi-Agent World Modeling from a Diffusion-Inspired Perspective

NeurIPS 2025poster

World models have recently attracted growing interest in Multi-Agent Reinforcement Learning (MARL) due to their ability to improve sample efficiency for policy learning. However, accurately modeling environments in MARL is challenging due to the exponentially large joint action space and highly unce…

Cited by 0SourcecodeScholar
2025

SAM-R1: Leveraging SAM for Reward Feedback in Multimodal Segmentation via Reinforcement Learning

NeurIPS 2025poster

Leveraging multimodal large models for image segmentation has become a prominent research direction. However, existing approaches typically rely heavily on manually annotated datasets that include explicit reasoning processes, which are costly and time-consuming to produce. Recent advances suggest t…

Cited by 0SourceScholar
2025

SUMO: Search-Based Uncertainty Estimation for Model-Based Offline Reinforcement Learning

AAAI 2025technical

The performance of offline reinforcement learning (RL) suffers from the limited size and quality of static datasets. Model-based offline RL addresses this issue by generating synthetic samples through a dynamics model to enhance overall performance. To evaluate the reliability of the generated sampl…

2025

SkillMimic: Learning Basketball Interaction Skills from Demonstrations

CVPR 2025highlight

Traditional reinforcement learning methods for human-object interaction (HOI) rely on labor-intensive, manually designed skill rewards that do not generalize well across different interactions. We introduce SkillMimic, a unified data-driven framework that fundamentally changes how agents learn inter…

2025

TCPO: Thought-Centric Preference Optimization for Effective Embodied Decision-making

EMNLP 2025

Using effective generalization capabilities of vision language models (VLMs) in context-specific dynamic tasks for embodied artificial intelligence remains a significant challenge. Although supervised fine-tuned models can better align with the real physical world, they still exhibit sluggish respon

Cited by 0SourcePDFScholar
2025

Taming Rectified Flow for Inversion and Editing

ICML 2025poster

Rectified-flow-based diffusion transformers like FLUX and OpenSora have demonstrated outstanding performance in the field of image and video generation. Despite their robust generative capabilities, these models often struggle with inversion inaccuracies, which could further limit their effectivenes…

2025

Towards Efficient LLM Grounding for Embodied Multi-Agent Collaboration

ACL 2025finding

Grounding the reasoning ability of large language models (LLMs) for embodied tasks is challenging due to the complexity of the physical world. Especially, LLM planning for multi-agent collaboration requires communication of agents or credit assignment as the feedback to re-adjust the proposed plans…

2025

VLP: Vision-Language Preference Learning for Embodied Manipulation

EMNLP 2025

Reward engineering is one of the key challenges in Reinforcement Learning (RL). Preference-based RL effectively addresses this issue by learning from human feedback. However, it is both time-consuming and expensive to collect human preference labels. In this paper, we propose a novel V ision- L angu

2025

World Models with Hints of Large Language Models for Goal Achieving

NAACL 2025long

Reinforcement learning struggles in the face of long-horizon tasks and sparse goals due to the difficulty in manual reward specification. While existing methods address this by adding intrinsic rewards, they may fail to provide meaningful guidance in long-horizon decision-making tasks with large sta…

Cited by 2SourcePDFScholar
2025

X-NeMo: Expressive Neural Motion Reenactment via Disentangled Latent Attention

ICLR 2025poster

We propose X-NeMo, a novel zero-shot diffusion-based portrait animation pipeline that animates a static portrait using facial movements from a driving video of a different individual. Our work first identifies the root causes of the limitations in prior approaches, such as identity leakage and diffi…

Cited by 0SourcePDFScholar
2024

BATON: Aligning Text-to-Audio Model Using Human Preference Feedback

IJCAI 2024poster

With the development of AI-Generated Content (AIGC), text-to-audio models are gaining widespread attention. However, it is challenging for these models to generate audio aligned with human preference due to the inherent information density of natural language and limited model understanding ability.…

2024

Bridging the Divide: Reconsidering Softmax and Linear Attention

NeurIPS 2024poster

Widely adopted in modern Vision Transformer designs, Softmax attention can effectively capture long-range visual information; however, it incurs excessive computational cost when dealing with high-resolution inputs. In contrast, linear attention naturally enjoys linear complexity and has great poten…

2024

Bridging the Gap: A Unified Video Comprehension Framework for Moment Retrieval and Highlight Detection

CVPR 2024poster

Video Moment Retrieval (MR) and Highlight Detection (HD) have attracted significant attention due to the growing demand for video analysis. Recent approaches treat MR and HD as similar video grounding problems and address them together with transformer-based architecture. However we observe that the…

2024

COVE: Unleashing the Diffusion Feature Correspondence for Consistent Video Editing

NeurIPS 2024poster

Video editing is an emerging task, in which most current methods adopt the pre-trained text-to-image (T2I) diffusion model to edit the source video in a zero-shot manner. Despite extensive efforts, maintaining the temporal consistency of edited videos remains challenging due to the lack of temporal…

2024

Chain of Generation: Multi-Modal Gesture Synthesis via Cascaded Conditional Control

AAAI 2024technical

This study aims to improve the generation of 3D gestures by utilizing multimodal information from human speech. Previous studies have focused on incorporating additional modalities to enhance the quality of generated gestures. However, these methods perform poorly when certain modalities are missing…

Cited by 14SourcePDFScholar
2024

Cross-Domain Policy Adaptation by Capturing Representation Mismatch

ICML 2024poster

It is vital to learn effective policies that can be transferred to different domains with dynamics discrepancies in reinforcement learning (RL). In this paper, we consider dynamics adaptation settings where there exists dynamics mismatch between the source domain and the target domain, and one can g…

2024

Cross-Modal Match for Language Conditioned 3D Object Grounding

AAAI 2024technical

Language conditioned 3D object grounding aims to find the object within the 3D scene mentioned by natural language descriptions, which mainly depends on the matching between visual and natural language. Considerable improvement in grounding performance is achieved by improving the multimodal fusion…

Cited by 9SourcePDFScholar
2024

Dual Mapping of 2D StyleGAN for 3D-Aware Image Generation and Manipulation (Student Abstract)

AAAI 2024technical

3D-aware GANs successfully solve the problem of 3D-consistency generation and furthermore provide a 3D shape of the generated object. However, the application of the volume renderer disturbs the disentanglement of the latent space, which makes it difficult to manipulate 3D-aware GANs and lowers the…

Cited by 0SourcePDFScholar
2024

Exploration and Anti-Exploration with Distributional Random Network Distillation

ICML 2024poster

Exploration remains a critical issue in deep reinforcement learning for an agent to attain high returns in unknown environments. Although the prevailing exploration Random Network Distillation (RND) algorithm has been demonstrated to be effective in numerous environments, it often needs more discrim…

2024

Exploring Multi-Modal Control in Music-Driven Dance Generation

ICASSP 2024accepted

Existing music-driven 3D dance generation methods mainly concentrate on high-quality dance generation, but lack sufficient control during the generation process. To address these issues, we propose a unified framework capable of generating high-quality dance movements and supporting multi-modal cont…

Cited by 0SourceScholar
2024

Follow Your Pose: Pose-Guided Text-to-Video Generation Using Pose-Free Videos

AAAI 2024technical

Generating text-editable and pose-controllable character videos have an imperious demand in creating various digital human. Nevertheless, this task has been restricted by the absence of a comprehensive dataset featuring paired video-pose captions and the generative prior models for videos. In this w…

2024

Lodge: A Coarse to Fine Diffusion Network for Long Dance Generation Guided by the Characteristic Dance Primitives

CVPR 2024poster

We propose Lodge a network capable of generating extremely long dance sequences conditioned on given music. We design Lodge as a two-stage coarse to fine diffusion architecture and propose the characteristic dance primitives that possess significant expressiveness as intermediate representations bet…

2024

MambaTalk: Efficient Holistic Gesture Synthesis with Selective State Space Models

NeurIPS 2024poster

Gesture synthesis is a vital realm of human-computer interaction, with wide-ranging applications across various fields like film, robotics, and virtual reality. Recent advancements have utilized the diffusion model to improve gesture synthesis. However, the high computational complexity of these t…

2024

MambaTree: Tree Topology is All You Need in State Space Model

NeurIPS 2024spotlight

The state space models, employing recursively propagated features, demonstrate strong representation capabilities comparable to Transformer models and superior efficiency. However, constrained by the inherent geometric constraints of sequences, it still falls short in modeling long-range dependencie…

Cited by 1SourcePDFScholar
2024

Mind the Interference: Retaining Pre-trained Knowledge in Parameter Efficient Continual Learning of Vision-Language Models

ECCV 2024poster

"This study addresses the Domain-Class Incremental Learning problem, a realistic but challenging continual learning scenario where both the domain distribution and target classes vary across tasks. To handle these diverse tasks, pre-trained Vision-Language Models (VLMs) are introduced for their stro…

2024

ODRL: A Benchmark for Off-Dynamics Reinforcement Learning

NeurIPS 2024poster

We consider off-dynamics reinforcement learning (RL) where one needs to transfer policies across different domains with dynamics mismatch. Despite the focus on developing dynamics-aware algorithms, this field is hindered due to the lack of a standard benchmark. To bridge this gap, we introduce ODRL,…

2024

PEARL: Zero-shot Cross-task Preference Alignment and Robust Reward Learning for Robotic Manipulation

ICML 2024poster

In preference-based Reinforcement Learning (RL), obtaining a large number of preference labels are both time-consuming and costly. Furthermore, the queried human preferences cannot be utilized for the new tasks. In this paper, we propose Zero-shot Cross-task Preference Alignment and Robust Reward Le…

Cited by 5SourcePDFScholar
2024

ROV6D: 6D Pose Estimation Benchmark Dataset for Underwater Remotely Operated Vehicles

RA-L 2024

Accurately localization between multi-robots is crucial for many underwater applications, such as tracking, convoying and subsea intervention tasks. 6D pose estimation is a fundamental task that enables precise object localization in 3D space with full six degrees of freedom. However, one critical c

Cited by 12SourceScholar
2024

Real-world Image Dehazing with Coherence-based Pseudo Labeling and Cooperative Unfolding Network

NeurIPS 2024spotlight

Real-world Image Dehazing (RID) aims to alleviate haze-induced degradation in real-world settings. This task remains challenging due to the complexities in accurately modeling real haze distributions and the scarcity of paired real-world data. To address these challenges, we first introduce a cooper…

2024

Robust Quadrupedal Locomotion via Risk-Averse Policy Learning

ICRA 2024poster

The robustness of legged locomotion is crucial for quadrupedal robots in challenging terrains. Recently, Reinforcement Learning (RL) has shown promising results in legged locomotion and various methods try to integrate privileged distillation, scene modeling, and external sensors to improve the gene…

Cited by 13SourceScholar
2024

SAM-E: Leveraging Visual Foundation Model with Sequence Imitation for Embodied Manipulation

ICML 2024poster

Acquiring a multi-task imitation policy in 3D manipulation poses challenges in terms of scene understanding and action prediction. Current methods employ both 3D representation and multi-view 2D representation to predict the poses of the robot’s end-effector. However, they still require a considerab…

Cited by 11SourcePDFScholar
2024

SEABO: A Simple Search-Based Method for Offline Imitation Learning

ICLR 2024poster

Offline reinforcement learning (RL) has attracted much attention due to its ability in learning from static offline datasets and eliminating the need of interacting with the environment. Nevertheless, the success of offline RL relies heavily on the offline transitions annotated with reward labels. I…

2024

STViT: Improving Self-Supervised Multi-Camera Depth Estimation with Spatial-Temporal Context and Adversarial Geometry Regularization (Student Abstract)

AAAI 2024technical

Multi-camera depth estimation has recently garnered significant attention due to its substantial practical implications in the realm of autonomous driving. In this paper, we delve into the task of self-supervised multi-camera depth estimation and propose an innovative framework, STViT, featuring sev…

Cited by 1SourcePDFScholar
2024

Strategic Preys Make Acute Predators: Enhancing Camouflaged Object Detectors by Generating Camouflaged Objects

ICLR 2024poster

Camouflaged object detection (COD) is the challenging task of identifying camouflaged objects visually blended into surroundings. Albeit achieving remarkable success, existing COD detectors still struggle to obtain precise results in some challenging cases. To handle this problem, we draw inspiratio…

2024

Text2Avatar: Text to 3d Human Avatar Generation with Codebook-Driven Body Controllable Attribute

ICASSP 2024accepted

Generating 3D human models directly from text helps reduce the cost and time of character modeling. However, achieving multi-attribute controllable and realistic 3D human avatar generation is still challenging due to feature coupling and the scarcity of realistic 3D human avatar datasets. To address…

Cited by 0SourceScholar
2024

UW-SDF: Exploiting Hybrid Geometric Priors for Neural SDF Reconstruction from Underwater Multi-view Monocular Images

IROS 2024

Due to the unique characteristics of underwater environments, accurate 3D reconstruction of underwater objects poses a challenging problem in tasks such as underwater exploration and mapping. Traditional methods that rely on multiple sensor data for 3D reconstruction are time-consuming and face chal

Cited by 2SourceScholar
2024

Using Human Feedback to Fine-tune Diffusion Models without Any Reward Model

CVPR 2024poster

Using reinforcement learning with human feedback (RLHF) has shown significant promise in fine-tuning diffusion models. Previous methods start by training a reward model that aligns with human preferences then leverage RL techniques to fine-tune the underlying models. However crafting an efficient re…

2023

CASR: Generating Complex Sequences with Autoregressive Self-Boost Refinement

ICLR 2023poster

There are sequence generation tasks where the best order to generate the target sequence is not left-to-right. For example, an answer to the Sudoku game, a structured code like s-expression, and even a logical natural language answer where the analysis may be generated after the decision. We define…

Cited by 0SourcePDFScholar
2023

Camouflaged Object Detection With Feature Decomposition and Edge Reconstruction

CVPR 2023poster

Camouflaged object detection (COD) aims to address the tough issue of identifying camouflaged objects visually blended into the surrounding backgrounds. COD is a challenging task due to the intrinsic similarity of camouflaged objects with the background, as well as their ambiguous boundaries. Existi…

Cited by 260SourcePDFScholar
2023

Data-Efficient Image Quality Assessment with Attention-Panel Decoder

AAAI 2023technical

Blind Image Quality Assessment (BIQA) is a fundamental task in computer vision, which however remains unresolved due to the complex distortion conditions and diversified image contents. To confront this challenge, we in this paper propose a novel BIQA pipeline based on the Transformer architecture,…

2023

Degradation-Resistant Unfolding Network for Heterogeneous Image Fusion

ICCV 2023poster

Heterogeneous image fusion (HIF) techniques aim to enhance image quality by merging complementary information from images captured by different sensors. Among these algorithms, deep unfolding network (DUN)-based methods achieve promising performance but still suffer from two issues: they lack a degr…

Cited by 32PDFScholar
2023

FLAG3D: A 3D Fitness Activity Dataset With Language Instruction

CVPR 2023poster

With the continuously thriving popularity around the world, fitness activity analytic has become an emerging research topic in computer vision. While a variety of new tasks and algorithms have been proposed recently, there are growing hunger for data resources involved in high-quality data, fine-gra…

2023

FineDance: A Fine-grained Choreography Dataset for 3D Full Body Dance Generation

ICCV 2023poster

Generating full-body and multi-genre dance sequences from given music is a challenging task, due to the limitations of existing datasets and the inherent complexity of the fine-grained hand motion and dance genres. To address these problems, we propose FineDance, which contains 14.6 hours of music-…

Cited by 58PDFcodeScholar
2023

GPT4Tools: Teaching Large Language Model to Use Tools via Self-instruction

NeurIPS 2023poster

This paper aims to efficiently enable Large Language Models (LLMs) to use multi-modal tools. The advanced proprietary LLMs, such as ChatGPT and GPT-4, have shown great potential for tool usage through sophisticated prompt engineering. Nevertheless, these models typically rely on prohibitive computat…

2023

MeGraph: Capturing Long-Range Interactions by Alternating Local and Hierarchical Aggregation on Multi-Scaled Graph Hierarchy

NeurIPS 2023poster

Graph neural networks, which typically exchange information between local neighbors, often struggle to capture long-range interactions (LRIs) within the graph. Building a graph hierarchy via graph pooling methods is a promising approach to address this challenge; however, hierarchical information pr…

2023

Multi-Modal Learning and Relaxation of Physical Conflict for an Exoskeleton Robot with Proprioceptive Perception

ICRA 2023poster

Exoskeleton robots provide assistive forces to suit the human subject via physical human-robot interaction. During the closely-coupled interaction, a mismatch between the wearer and the robot may result in physical conflict, which could affect assistance efficiency or even compromise safety. Therefo…

Cited by 5SourceScholar
2023

SOC: Semantic-Assisted Object Cluster for Referring Video Object Segmentation

NeurIPS 2023poster

This paper studies referring video object segmentation (RVOS) by boosting video-level visual-linguistic alignment. Recent approaches model the RVOS task as a sequence prediction problem and perform multi-modal interaction as well as segmentation for each frame separately. However, the lack of a glob…

2023

SemanticAC: Semantics-Assisted Framework for Audio Classification

ICASSP 2023accepted

In this paper, we propose SemanticAC, a semantics-assisted framework for Audio Classification to better leverage the semantic information. Unlike conventional audio classification methods that treat class labels as discrete vectors, we employ a language model to extract abundant semantics from label…

Cited by 0SourceScholar
2023

Towards Realizing the Value of Labeled Target Samples: A Two-Stage Approach for Semi-Supervised Domain Adaptation

ICASSP 2023accepted

Semi-Supervised Domain Adaptation (SSDA) is a recently emerging research topic that extends from the widely-investigated Unsupervised Domain Adaptation (UDA) by further having a few target samples labeled, i.e., the model is trained with labeled source samples, unlabeled target samples as well as a…

Cited by 0SourceScholar
2023

Weakly-Supervised Concealed Object Segmentation with SAM-based Pseudo Labeling and Multi-scale Feature Grouping

NeurIPS 2023poster

Weakly-Supervised Concealed Object Segmentation (WSCOS) aims to segment objects well blended with surrounding environments using sparsely-annotated data for model training. It remains a challenging task since (1) it is hard to distinguish concealed objects from the background due to the intrinsic s…

Cited by 132SourcePDFScholar
2022

Double Check Your State Before Trusting It: Confidence-Aware Bidirectional Offline Model-Based Imagination

NeurIPS 2022accept

The learned policy of model-free offline reinforcement learning (RL) methods is often constrained to stay within the support of datasets to avoid possible dangerous out-of-distribution actions or states, making it challenging to handle out-of-support region. Model-based RL methods offer a richer dat…

2022

Efficient Continuous Control with Double Actors and Regularized Critics

AAAI 2022technical

How to obtain good value estimation is a critical problem in Reinforcement Learning (RL). Current value estimation methods in continuous control, such as DDPG and TD3, suffer from unnecessary over- or under- estimation. In this paper, we explore the potential of double actors, which has been neglect…

2022

Hierarchical Learning and Control for In-Hand Micromanipulation Using Multiple Laser-Driven Micro-Tools

IROS 2022poster

Laser-driven micro-tools are formulated by treating highly-focused laser beams as actuators, to control the tool's motion to contact then manipulate a micro object, which allows it to manipulate opaque micro objects, or large cells without causing photodamage. However, most existing laser-driven too…

Cited by 1SourceScholar
2022

Mildly Conservative Q-Learning for Offline Reinforcement Learning

NeurIPS 2022accept

Offline reinforcement learning (RL) defines the task of learning from a static logged dataset without continually interacting with the environment. The distribution shift between the learned policy and the behavior policy makes it necessary for the value function to stay conservative such that out-o…

2022

Neural Capture of Animatable 3D Human from Monocular Video

ECCV 2022poster

"We present a novel paradigm of building an animatable 3D human representation from a monocular video input, such that it can be rendered in any unseen poses and views. Our method is based on a dynamic Neural Radiance Field (NeRF) rigged by a mesh-based parametric 3D human model serving as a geometr…

Cited by 28SourcePDFScholar
2022

OrdinalCLIP: Learning Rank Prompts for Language-Guided Ordinal Regression

NeurIPS 2022accept

This paper presents a language-powered paradigm for ordinal regression. Existing methods usually treat each rank as a category and employ a set of weights to learn these concepts. These methods are easy to overfit and usually attain unsatisfactory performance as the learned concepts are mainly deriv…

2022

Rethinking Goal-Conditioned Supervised Learning and Its Connection to Offline RL

ICLR 2022poster

Solving goal-conditioned tasks with sparse rewards using self-supervised learning is promising because of its simplicity and stability over current reinforcement learning (RL) algorithms. A recent work, called Goal-Conditioned Supervised Learning (GCSL), provides a new learning framework by iterativ…

2022

ScalableViT: Rethinking the Context-Oriented Generalization of Vision Transformer

ECCV 2022poster

"The vanilla self-attention mechanism inherently relies on pre-defined and steadfast computational dimensions. Such inflexibility restricts it from possessing context-oriented generalization that can bring more contextual cues and graphic representations. To mitigate this issue, we propose a Scalabl…

2021

Efficient Face Manipulation Via Deep Feature Disentanglement And Reintegration Net

ICASSP 2021accepted

Deep neural networks (DNNs) have been widely used in facial manipulation. Existing methods focus on training deeper networks in indirect supervision ways (e.g., feature constraint), or in unsupervised ways (e.g., cycle-consistency loss) due to the lack of ground-truth face images for manipulated out…

Cited by 1SourceScholar
2021

Frequency-Aware Spatiotemporal Transformers for Video Inpainting Detection

ICCV 2021poster

In this paper, we propose a frequency-aware spatiotemporal transformers for deep In this paper, we propose a Frequency-Aware Spatiotemporal Transformer (FAST) for video inpainting detection, which aims to simultaneously mine the traces of video inpainting from spatial, temporal, and frequency domain…

Cited by 36PDFScholar
2021

Universal and Flexible Optical Aberration Correction Using Deep-Prior Based Deconvolution

ICCV 2021poster

High quality imaging usually requires bulky and expensive lenses to compensate geometric and chromatic aberrations. This poses high constraints on the optical hash or low cost applications. Although one can utilize algorithmic reconstruction to remove the artifacts of low-end lenses, the degeneratio…

Cited by 30PDFcodeScholar
2021

Unsupervised Multimodal Image Registration with Adaptative Gradient Guidance

ICASSP 2021accepted

Multimodal image registration (MIR) is a fundamental procedure in many image-guided therapies. Recently, unsupervised learning-based methods have demonstrated promising performance over accuracy and efficiency in deformable image registration. However, the estimated deformation fields of the existin…

Cited by 0SourceScholar
2020

4D Association Graph for Realtime Multi-Person Motion Capture Using Multiple Video Cameras

CVPR 2020oral

his paper contributes a novel realtime multi-person motion capture algorithm using multiview video inputs. Due to the heavy occlusions and closely interacting motions in each view, joint optimization on the multiview images and multiple temporal frames is indispensable, which brings up the essential…

Cited by 104PDFcodeScholar