← Search

Yang Zhou

134 accepted papers

2026

$\pi^3$: Permutation-Equivariant Visual Geometry Learning

ICLR 2026poster

We introduce $\pi^3$, a feed-forward neural network that offers a novel approach to visual geometry reconstruction, breaking the reliance on a conventional fixed reference view. Previous methods often anchor their reconstructions to a designated viewpoint, an inductive bias that can lead to instabil…

Cited by 0SourcecodeScholar
2026

Augmented Radiance Field: A General Framework for Enhanced Gaussian Splatting

ICLR 2026poster

Due to the real-time rendering performance, 3D Gaussian Splatting (3DGS) has emerged as the leading method for radiance field reconstruction. However, its reliance on spherical harmonics for color encoding inherently limits its ability to separate diffuse and specular components, making it challengi…

Cited by 0SourceScholar
2026

Automatic Dialectic Jailbreak: A Framework for Generating Effective Jailbreak Strategies

ICLR 2026poster

Large language models (LLMs) can be jailbroken to produce malicious or unethical content with embedded jailbreaking prompts. Unfortunately, current jailbreak attack techniques suffer from adaptability issues due to reliance on the fixed evaluation models and incapability problems of surviving from a…

Cited by 0SourceScholar
2026

Breaking the Exploration Bottleneck: Rubric-Scaffolded Reinforcement Learning for General LLM Reasoning

ICML 2026poster

Recent advances in Large Language Models (LLMs) have underscored the potential of Reinforcement Learning (RL) to facilitate the emergence of reasoning capabilities. Despite the encouraging results, a fundamental dilemma persists as RL improvement relies on learning from high-quality samples, yet the…

Cited by 0SourceScholar
2026

CUE: Concept-Aware Multi-Label Expansion to Mitigate Concept Confusion in Long-Tailed Learning

CVPR 2026

Long-tailed distributions are common in real-world recognition tasks, where a few head classes have many samples while most tail classes have very few. Recently, fine-tuning foundation models for long-tailed learning has gained attention due to their excellent performance. However, most existing met

Cited by 0SourcecodeScholar
2026

ConServe: Fine-Grained GPU Harvesting for LLM Online and Offline Co-Serving

ICML 2026poster

Large language model (LLM) serving demands low latency and high throughput, but high load variability leads to significant GPU utilization. In this paper, we identify a synergetic but overlooked opportunity to co-serve latency-critical online requests alongside *latency-tolerant offline* tasks, whic…

Cited by 0SourceScholar
2026

CyPortQA: Benchmarking Multimodal Large Language Models for Cyclone Preparedness in Port Operation

AAAI 2026technical

As tropical cyclones intensify and track forecasts become increasingly uncertain, U.S. ports face heightened supply-chain risk under extreme weather conditions. Port operators need to rapidly synthesize diverse multimodal forecast products, such as probabilistic wind maps, track cones, and official

Cited by 0SourcePDFScholar
2026

DINO Eats CLIP: Adapting Beyond Knowns for Open-set 3D Object Retrieval

CVPR 2026

Vision foundation models have shown great promise for open-set 3D object retrieval (3DOR) through efficient adaptation to multi-view images. Leveraging semantically aligned latent space, previous work typically adapts the CLIP encoder to build view-based 3D descriptors. Despite CLIP's strong general

Cited by 0SourceScholar
2026

DrivingGen: A Comprehensive Benchmark for Generative Video World Models in Autonomous Driving

ICLR 2026poster

Video generation models, as one form of world models, has emerged as one of the most exciting frontiers in AI, promising agents the ability to imagine the future by modeling the temporal evolution of complex scenes. In autonomous driving, this vision gives rise to driving world models—generative si…

Cited by 0SourceScholar
2026

EditCtrl: Disentangled Local and Global Control for Real-Time Generative Video Editing

CVPR 2026

High-fidelity generative video editing has seen significant quality improvements by leveraging pre-trained video foundation models. However, their computational cost is a major bottleneck, as they are often designed to inefficiently process the full video context regardless of the inpainting mask's

Cited by 0SourcecodeScholar
2026

Evaluating LLMs When They Do Not Know the Answer: Statistical Evaluation of Mathematical Reasoning via Comparative Signals

ICML 2026poster

Evaluating mathematical reasoning in LLMs is constrained by limited benchmark sizes and inherent model stochasticity, yielding high-variance accuracy estimates and unstable rankings across platforms. On difficult problems, an LLM may fail to produce a correct final answer, yet still provide reliable…

Cited by 0SourceScholar
2026

FracSegmentator: Fracture Instance Segmentation with Trauma-Prior-Guided Contrastive Learning

AAAI 2026technical

Fracture injuries often lead to complex bone fragmentations, posing significant challenges for accurate segmentation in surgical planning and trauma assessment. Manual annotation of each fragment is time-consuming and inconsistent, while existing automated methods often fail to separate individual f

Cited by 0SourcePDFScholar
2026

FusedRec: Fused Embedding Communication for Distributed Recommendation Training on GPUs

AAAI 2026technical

Recent years have witnessed the wide adoption of deep learning recommendation models (DLRMs) for many online services. Unlike traditional DNN training, DLRMs leverage massive embeddings to represent sparse features, which are stored in distributed GPUs following the model parallel paradigm. Existing

Cited by 0SourcePDFScholar
2026

Generalizing Linear Autoencoder Recommenders with Decoupled Expected Quadratic Loss

ICLR 2026poster

Linear autoencoders (LAEs) have gained increasing popularity in recommender systems due to their simplicity and strong empirical performance. Most LAE models, including the Emphasized Denoising Linear Autoencoder (EDLAE) introduced by (Steck, 2020), use quadratic loss during training. However, the o…

Cited by 0SourceScholar
2026

Graph Neural Planning and Predictive Control for Multi-Robot Communication-Constrained Unlabeled Motion Planning

ICRA 2026poster

The multi-robot unlabeled motion planning problem of concurrently assigning robots to goals and generating safe trajectories is central in many collaborative tasks. Recent Graph Neural Network methods offer scalable decentralized solutions but rely on simplified dynamics and simulation environments,…

2026

HoLoArm: Deformable Arms for Collision-Tolerant Quadrotor Flight

ICRA 2026poster

The increasing use of drones in human-centric applications highlights the need for designs that can survive collisions and recover rapidly, minimizing risks to both humans and the environment. We present HoLoArm, a quadrotor with compliant arms inspired by the nodus structure of dragonfly wings. Thi…

2026

Jackpot: Align Actor-Policy Distribution for scalable and stable RL for LLM

ICLR 2026poster

Reinforcement learning (RL) has become an increasingly important paradigm for improving large language models (LLMs) on alignment, reasoning, and coding tasks, yet it remains extremely costly. The majority of training time is spent on rollouts. Allowing actor and policy distributions to differ could…

Cited by 0SourcecodeScholar
2026

K-Prism: A Knowledge-Guided and Prompt Integrated Universal Medical Image Segmentation Model

ICLR 2026poster

Medical image segmentation is fundamental to clinical decision-making, yet existing models remain fragmented. They are usually trained on single knowledge sources and specific to individual tasks, modalities, or organs. This fragmentation contrasts sharply with clinical practice, where experts seaml…

Cited by 0SourcecodeScholar
2026

Mitigating the Modality Gap in Vision–Language Models with Fractal Spectral Geometry

ICML 2026poster

Vision–language models such as CLIP embed images and text into a shared space, but still suffer from a modality gap, where image and text features cluster separately and nearest neighbors are dominated by same-modality rather than true cross-modal matches. Existing works alleviate the modality gap b…

Cited by 0SourceScholar
2026

NeuroMesh: A Unified Neural Inference Framework for Decentralized Multi-Robot Collaboration

RA-L 2026

Deploying learned multi-robot models on heterogeneous robots remains challenging due to hardware heterogeneity, communication constraints, and the lack of a unified execution stack. This paper presents NeuroMesh, a multi-domain, cross-platform, and modular decentralized neural inference framework th

Cited by 0SourceScholar
2026

Note2Chat: Improving LLMs for Multi-Turn Clinical History Taking Using Medical Notes

AAAI 2026technical

Effective clinical history taking is a foundational yet underexplored component of clinical reasoning. While large language models (LLMs) have shown promise on static benchmarks, they often fall short in dynamic, multi-turn diagnostic settings that require iterative questioning and hypothesis refine

Cited by 0SourcePDFScholar
2026

OWOD-FSL: Open-World Object Detection Via Few-Shot Learning and Dynamic Prototypes

ICRA 2026poster

Open-World Object Detection (OWOD) presents a critical challenge for modern computer vision systems: detecting known classes, identifying unknown objects, and incrementally learning to recognize them over time. However, current approaches have two fundamental limitations: (1) the fixed-dimensional c…

Cited by 0Scholar
2026

OmniWorld: A Multi-Domain and Multi-Modal Dataset for 4D World Modeling

ICLR 2026poster

The field of 4D world modeling—aiming to jointly capture spatial geometry and temporal dynamics—has witnessed remarkable progress in recent years, driven by advances in large-scale generative models and multimodal learning. However, the development of truly general 4D world models remains fundamen…

Cited by 0SourcecodeScholar
2026

Query-Based Adaptive Aggregation for Multi-Dataset Joint Training Toward Universal Visual Place Recognition

ICRA 2026poster

Deep learning methods for Visual Place Recognition (VPR) have advanced significantly, largely driven by large-scale datasets. However, most existing approaches are trained on a single dataset, which can introduce dataset-specific inductive biases and limit model generalization. While multi-dataset j…

2026

Replay Failures as Successes: Sample-Efficient Reinforcement Learning for Instruction Following

ICML 2026poster

Reinforcement Learning (RL) has shown promise for aligning Large Language Models (LLMs) to follow instructions with various constraints. Despite the encouraging results, RL improvement inevitably relies on sampling successful, high-quality responses; however, the initial model often struggles to gen…

Cited by 4SourceScholar
2026

Structured Multi-step Jailbreaking under a Hamiltonian Generative Formulation

ICML 2026poster

Recent work shows that even safety aligned large language models (LLM) can be pushed into unsafe behavior by carefully crafted jailbreak prompts. Existing jailbreaking attack methods often rely on disfluent or incoherent prompts, which limit their success and make them easy to detect. We introduce S…

Cited by 0SourceScholar
2026

UVLM: Benchmarking Video Language Model for Underwater World Understanding

AAAI 2026technical

Recently, video-language models (VidLMs) have gained widespread attention and adoption. However, existing works primarily focus on terrestrial scenarios, overlooking the highly demanding application needs of underwater observation. To overcome this gap, we introduce UVLM, an under water observation

Cited by 0SourcePDFScholar
2026

WinT3R: Window-Based Streaming Reconstruction with Camera Token Pool

ICLR 2026poster

We present WinT3R, a feed-forward reconstruction model capable of online prediction of precise camera poses and high-quality point maps. Previous methods suffer from a trade-off between reconstruction quality and real-time performance. To address this, we first introduce a sliding window mechanism t…

Cited by 0SourcecodeScholar
2025

A Robust Method to Discover Causal or Anticausal Relation

ICLR 2025poster

Understanding whether the data generative process follows causal or anticausal relations is important for many applications. Existing causal discovery methods struggle with high-dimensional perceptual data such as images. Moreover, they require well-labeled data, which may not be feasible due to mea…

Cited by 0SourcePDFScholar
2025

Act Only When It Pays: Efficient Reinforcement Learning for LLM Reasoning via Selective Rollouts

NeurIPS 2025poster

Reinforcement learning, such as PPO and GRPO, has powered recent breakthroughs in LLM reasoning. Scaling rollout to sample more prompts enables models to selectively use higher-quality data for training, which can stabilize RL training and improve model performance, but at the cost of significant co…

Cited by 0SourcecodeScholar
2025

Aether: Geometric-Aware Unified World Modeling

ICCV 2025poster

The integration of geometric reconstruction and generative modeling remains a critical challenge in developing AI systems capable of human-like spatial reasoning. This paper proposes Aether, a unified framework that enables geometry-aware reasoning in world models by jointly optimizing three core ca…

Cited by 0SourcePDFScholar
2025

Attention Distillation: A Unified Approach to Visual Characteristics Transfer

CVPR 2025poster

Recent advances in generative diffusion models have shown a notable inherent understanding of image style and semantics. In this paper, we leverage the self-attention features from pretrained diffusion networks to transfer the visual characteristics from a reference to generated images. Unlike previ…

2025

Bridging Symmetry and Robustness: On the Role of Equivariance in Enhancing Adversarial Robustness

NeurIPS 2025spotlight

Adversarial examples reveal critical vulnerabilities in deep neural networks by exploiting their sensitivity to imperceptible input perturbations. While adversarial training remains the predominant defense strategy, it often incurs significant computational cost and may compromise clean-data accurac…

Cited by 0SourceScholar
2025

CLAP: A Closed-Loop Diffusion Transformer Action Foundation Model for Robotic Manipulation

IROS 2025

The development of large Vision-Language-Action (VLA) models has enhanced the robot’s ability to manipulate objects in unseen scenarios based on language instructions. While existing VLAs have demonstrated promise in various scenarios, they still struggle with effective multi-modal data feature extr

Cited by 1SourceScholar
2025

Causal-Planner: Causal Interaction Disentangling with Episodic Memory Gating for Autonomous Planning

IROS 2025

Autonomous vehicle trajectory planning faces significant challenges in dynamic traffic environments due to the complex and mixed causal relationships between critical scene elements (e.g., pedestrians, vehicles, road markings) and safe decision-making. To identify the causal factors influencing plan

Cited by 0SourcecodeScholar
2025

DMesh++: An Efficient Differentiable Mesh for Complex Shapes

ICCV 2025poster

Recent probabilistic methods for 3D triangular meshes capture diverse shapes by differentiable mesh connectivity, but face high computational costs with increased shape details. We introduce a new differentiable mesh processing method that addresses this challenge and efficiently handles meshes with…

2025

Describe, Adapt and Combine: Empowering CLIP Encoders for Open-set 3D Object Retrieval

ICCV 2025poster

Open-set 3D object retrieval (3DOR) is an emerging task aiming to retrieve 3D objects of unseen categories beyond the training set. Existing methods typically utilize all modalities (i.e., voxels, point clouds, multi-view images) and train specific backbones before fusion. However, they still strugg…

2025

FPE2M2: Approaching Lossless and Efficient Quantization with Native Floating Point

ACL 2025finding

Auto-regressive decoding is a memory-bound job, meaning decoding inference performance is limited by the bandwidth rather than the computational capabilities of the GPU. Weight-only quantization is a promising method to address the memory-bound limitations. Previous studies have followed one of two…

Cited by 0SourcePDFScholar
2025

Faster Annotation for Elevation-Guided Flood Extent Mapping by Consistency-Enhanced Active Learning

IJCAI 2025

Flood extent mapping is crucial for disaster response and damage assessment. While Earth imagery and terrain data (in the form of DEM) are now readily available, there are few flood annotation data for training machine learning models, which hinders the automated mapping of flooded areas. We propose

2025

Flexible, Efficient, and Stable Adversarial Attacks on Machine Unlearning

ICML 2025poster

Machine unlearning (MU) aims to remove the influence of specific data points from trained models, enhancing compliance with privacy regulations. However, the vulnerability of basic MU models to malicious unlearning requests in adversarial learning environments has been largely overlooked. Existing a…

2025

Free-viewpoint Human Animation with Pose-correlated Reference Selection

CVPR 2025highlight

Diffusion-based human animation aims to animate a human character based on a source human image as well as driving signals such as a sequence of poses. Leveraging the generative capacity of diffusion model, existing approaches are able to generate high-fidelity poses, but struggle with significant v…

Cited by 1SourcePDFScholar
2025

GSM-$\infty$: How Do your LLMs Behave over Infinitely Increasing Reasoning Complexity and Context Length?

ICML 2025poster

Recently, long-context large language models (LLMs) have shown strong performance in information retrieval and long-document QA. However, to tackle the most challenging intellectual problems, LLMs must reason effectively in long and complex contexts (e.g., frontier mathematical research). Studying h…

Cited by 0SourcePDFScholar
2025

HiSplat: Hierarchical 3D Gaussian Splatting for Generalizable Sparse-View Reconstruction

ICLR 2025poster

Reconstructing 3D scenes from multiple viewpoints is a fundamental task in stereo vision. Recently, advances in generalizable 3D Gaussian Splatting have enabled high-quality novel view synthesis for unseen scenes from sparse input views by feed-forward predicting per-pixel Gaussian parameters withou…

2025

Instance-Level Video Depth in Groups Beyond Occlusions

ICCV 2025poster

Depth estimation in dynamic, multi-object scenes remains a major challenge, especially under severe occlusions. Existing monocular models, including foundation models, struggle with instance-wise depth consistency due to their reliance on global regression. We tackle this problem from two key aspect…

Cited by 0SourcePDFScholar
2025

Jury-and-Judge Chain-of-Thought for Uncovering Toxic Data in 3D Visual Grounding

NeurIPS 2025poster

3D Visual Grounding (3DVG) faces persistent challenges due to coarse scene-level observations and logically inconsistent annotations, which introduce ambiguities that compromise data quality and hinder effective model supervision. To address these challenges, we introduce Refer-Judge, a novel framew…

Cited by 0SourcecodeScholar
2025

M2Edit: Locate and Edit Multi-Granularity Knowledge in Multimodal Large Language Model

EMNLP 2025

Multimodal knowledge editing is an important method for modifying outdated or incorrect knowledge in Multimodal Large Language Models (MLLMs). However, existing datasets for multimodal knowledge editing lack multi-granularity knowledge. In this paper, we present a more realistic dataset called M2Edi

2025

MagicPIG: LSH Sampling for Efficient LLM Generation

ICLR 2025spotlight

Large language models (LLMs) with long context windows have gained significant attention. However, the KV cache, stored to avoid re-computation, becomes a bottleneck. Various dynamic sparse or TopK-based attention approximation methods have been proposed to leverage the common insight that attention…

2025

Move-in-2D: 2D-Conditioned Human Motion Generation

CVPR 2025poster

Generating realistic human videos remains a challenging task, with the most effective methods currently relying on a human motion sequence as a control signal. Existing approaches often use existing motion extracted from other videos, which restricts applications to specific motion types and global…

2025

PAC-Bayes Bounds for Multivariate Linear Regression and Linear Autoencoders

NeurIPS 2025poster

Linear Autoencoders (LAEs) have shown strong performance in state-of-the-art recommender systems. However, this success remains largely empirical, with limited theoretical understanding. In this paper, we investigate the generalizability -- a theoretical measure of model performance in statistical…

Cited by 0SourceScholar
2025

Privacy-Preserving V2X Collaborative Perception Integrating Unknown Collaborators

AAAI 2025technical

Vehicle-to-everything (V2X) collaborative perception has recently gained increasing attention in autonomous driving due to its ability to enhance scene understanding by integrating information from other collaborators, e.g. vehicles or infrastructure. Existing algorithms usually share deep features…

Cited by 0SourcePDFScholar
2025

Rewind and Render: Towards Factually Accurate Text-to-Video Generation with Distilled Knowledge Retrieval

AAAI 2025technical

Text-to-Video (T2V) models, despite recent advancements, struggle with factual accuracy, especially for knowledge-dense content. We introduce FACT-V (Factual Accuracy in Content Translation to Video), a system integrating multi-source knowledge retrieval into T2V pipelines. FACT-V offers two key ben…

Cited by 0SourcePDFScholar
2025

SeRL: Self-play Reinforcement Learning for Large Language Models with Limited Data

NeurIPS 2025poster

Recent advances have demonstrated the effectiveness of Reinforcement Learning (RL) in improving the reasoning capabilities of Large Language Models (LLMs). However, existing works inevitably rely on high-quality instructions and verifiable rewards for effective training, both of which are often diff…

Cited by 0SourcecodeScholar
2025

Self-Evolutionary Large Language Models Through Uncertainty-Enhanced Preference Optimization

AAAI 2025technical

Iterative preference optimization has recently become one of the de-facto training paradigms for large language models (LLMs), but the performance is still underwhelming due to too much noisy preference data yielded in the loop. To combat this issue, we present an Uncertainty-enhanced Preference Opt…

2025

ShotVL: Human-Centric Highlight Frame Retrieval via Language Queries

AAAI 2025technical

Existing research on human-centric video understanding typically focuses on analyzing specific moments or entire videos. However, many applications require higher precision at the frame level. In this work, we propose a novel task, BestShot, which aims to locate highlight frames within human-centric…

2025

SmartPretrain: Model-Agnostic and Dataset-Agnostic Representation Learning for Motion Prediction

ICLR 2025poster

Predicting the future motion of surrounding agents is essential for autonomous vehicles (AVs) to operate safely in dynamic, human-robot-mixed environments. However, the scarcity of large-scale driving datasets has hindered the development of robust and generalizable motion prediction models, limitin…

2025

The Curse of Multi-Modalities: Evaluating Hallucinations of Large Multimodal Models across Language, Visual, and Audio

NeurIPS 2025poster

Recent advancements in large multimodal models (LMMs) have significantly enhanced performance across diverse tasks, with ongoing efforts to further integrate additional modalities such as video and audio. However, most existing LMMs remain vulnerable to hallucinations, the discrepancy between the fa…

Cited by 0SourcecodeScholar
2025

Treasures in Discarded Weights for LLM Quantization

AAAI 2025technical

In recent years, large language models (LLMs) have developed rapidly and revolutionized natural language processing. However, high storage overhead and computing costs limit LLM deployment in resource-constrained environments. Quantization algorithms can effectively compress LLMs and accelerate infe…

Cited by 0SourcePDFScholar
2025

Unsupervised Anomaly Detection Improves Imitation Learning for Autonomous Racing

IROS 2025

Imitation Learning (IL) has shown significant promise in autonomous driving, but its performance heavily depends on the quality of training data. Noisy or corrupted sensor inputs can degrade learned policies, leading to unsafe behavior. This paper presents an unsupervised anomaly detection approach

Cited by 1SourceScholar
2025

VideoGigaGAN: Towards Detail-rich Video Super-Resolution

CVPR 2025poster

Video super-resolution (VSR) models achieve temporal consistency but often produce blurrier results than their image-based counterparts due to limited generative capacity. This prompts the question: can we adapt a generative image upsampler for VSR while preserving temporal consistency? We introduce…

Cited by 17SourcePDFScholar
2025

Visual Persona: Foundation Model for Full-Body Human Customization

CVPR 2025poster

We introduce Visual Persona, a foundation model for text-to-image full-body human customization that, given a single in-the-wild human image, generates diverse images of the individual guided by text descriptions. Unlike prior methods that focus solely on preserving facial identity, our approach cap…

Cited by 0SourcePDFScholar
2025

Visual Textualization for Image Prompted Object Detection

ICCV 2025poster

We propose VisTex-OVLM, a novel image prompted object detection method that introduces visual textualization ---- a process that projects a few visual exemplars into the text feature space to enhance Object-level Vision-Language Models' (OVLMs) capability in detecting rare categories that are diffic…

Cited by 0SourcePDFScholar
2025

VividFace: A Robost and High-Fidelity Video Face Swapping Framework

NeurIPS 2025poster

Video face swapping has seen increasing adoption in diverse applications, yet existing methods primarily trained on static images struggle to address temporal consistency and complex real-world scenarios. To overcome these limitations, we propose the first video face swapping framework, VividFace,…

Cited by 0SourceScholar
2024

ActAnywhere: Subject-Aware Video Background Generation

NeurIPS 2024poster

We study a novel problem to automatically generate video background that tailors to foreground subject motion. It is an important problem for the movie industry and visual effects community, which traditionally requires tedious manual efforts to solve. To this end, we propose ActAnywhere, a video di…

2024

BenchX: A Unified Benchmark Framework for Medical Vision-Language Pretraining on Chest X-Rays

NeurIPS 2024poster

Medical Vision-Language Pretraining (MedVLP) shows promise in learning generalizable and transferable visual representations from paired and unpaired medical images and reports. MedVLP can provide useful features to downstream tasks and facilitate adapting task-specific models to new setups using fe…

2024

CoPeD-Advancing Multi-Robot Collaborative Perception: A Comprehensive Dataset in Real-World Environments

RA-L 2024

In the past decade, although single-robot perception has made significant advancements, the exploration of multi-robot collaborative perception remains largely unexplored. This involves fusing compressed, intermittent, limited, heterogeneous, and asynchronous environmental information across multipl

Cited by 9SourcecodeScholar
2024

Customize-A-Video: One-Shot Motion Customization of Text-to-Video Diffusion Models

ECCV 2024poster

"Image customization has been extensively studied in text-to-image (T2I) diffusion models, leading to impressive outcomes and applications. With the emergence of text-to-video (T2V) diffusion models, its temporal counterpart, motion customization, has not yet been well investigated. To address the c…

2024

DMesh: A Differentiable Mesh Representation

NeurIPS 2024poster

We present a differentiable representation, DMesh, for general 3D triangular meshes. DMesh considers both the geometry and connectivity information of a mesh. In our design, we first get a set of convex tetrahedra that compactly tessellates the domain based on Weighted Delaunay Triangulation (WDT),…

2024

DVPE: Divided View Position Embedding for Multi-View 3D Object Detection

IJCAI 2024poster

Sparse query-based paradigms have achieved significant success in multi-view 3D detection for autonomous vehicles. Current research faces challenges in balancing between enlarging receptive fields and reducing interference when aggregating multi-view features. Moreover, different poses of cameras pr…

2024

EvaNet: Elevation-Guided Flood Extent Mapping on Earth Imagery

IJCAI 2024poster

Accurate and timely mapping of flood extent from high resolution satellite imagery plays a crucial role in disaster management such as damage assessment and relief activities. However, current state-of-the-art solutions are based on U-Net, which cannot segment the flood pixels accurately due to the…

2024

FedASMU: Efficient Asynchronous Federated Learning with Dynamic Staleness-Aware Model Update

AAAI 2024technical

As a promising approach to deal with distributed data, Federated Learning (FL) achieves major advancements in recent years. FL enables collaborative model training by exploiting the raw data dispersed in multiple edge devices. However, the data is generally non-independent and identically distribute…

Cited by 33SourcePDFScholar
2024

Fisher Information-based Efficient Curriculum Federated Learning with Large Language Models

EMNLP 2024main

As a promising paradigm to collaboratively train models with decentralized data, Federated Learning (FL) can be exploited to fine-tune Large Language Models (LLMs). While LLMs correspond to huge size, the scale of the training data significantly increases, which leads to tremendous amounts of comput…

Cited by 1SourcePDFScholar
2024

Generating Non-Stationary Textures using Self-Rectification

CVPR 2024poster

This paper addresses the challenge of example-based non-stationary texture synthesis. We introduce a novel two-step approach wherein users first modify a reference texture using standard image editing tools yielding an initial rough target for the synthesis. Subsequently our proposed method termed "…

2024

HARIVO: Harnessing Text-to-Image Models for Video Generation

ECCV 2024poster

"We present a method to create diffusion-based video models from pretrained Text-to-Image (T2I) models. Recently, AnimateDiff proposed freezing the T2I model while only training temporal layers. We advance this method by proposing a unique architecture, incorporating a mapping network and frame-wise…

2024

HexGen: Generative Inference of Large Language Model over Heterogeneous Environment

ICML 2024poster

Serving generative inference of the large language model is a crucial component of contemporary AI applications. In this paper, our focus lies in deploying such services in a heterogeneous and cross-datacenter setting to mitigate the substantial inference costs typically associated with a single cen…

2024

In-Hand 3D Object Reconstruction from a Monocular RGB Video

AAAI 2024technical

Our work aims to reconstruct a 3D object that is held and rotated by a hand in front of a static RGB camera. Previous methods that use implicit neural representations to recover the geometry of a generic hand-held object from multi-view images achieved compelling results in the visible part of the o…

2024

LRM: Large Reconstruction Model for Single Image to 3D

ICLR 2024oral

We propose the first Large Reconstruction Model (LRM) that predicts the 3D model of an object from a single input image within just 5 seconds. In contrast to many previous methods that are trained on small-scale datasets such as ShapeNet in a category-specific fashion, LRM adopts a highly scalable t…

Cited by 411SourcePDFScholar
2024

Leveraging Frame Affinity for sRGB-to-RAW Video De-rendering

CVPR 2024poster

Unprocessed RAW video has shown distinct advantages over sRGB video in video editing and computer vision tasks. However capturing RAW video is challenging due to limitations in bandwidth and storage. Various methods have been proposed to address similar issues in single image RAW capture through de-…

Cited by 2SourcePDFScholar
2024

SIRIUS : Contexual Sparisty with Correction for Efficient LLMs

NeurIPS 2024poster

With the blossom of large language models (LLM), inference efficiency becomes increasingly important. Various approximate methods are proposed to reduce the cost at inference time. Contextual Sparsity (CS) is appealing for its training-free nature and its ability to reach a higher compression ratio…

2024

SmartRefine: A Scenario-Adaptive Refinement Framework for Efficient Motion Prediction

CVPR 2024poster

Predicting the future motion of surrounding agents is essential for autonomous vehicles (AVs) to operate safely in dynamic human-robot-mixed environments. Context information such as road maps and surrounding agents' states provides crucial geometric and semantic information for motion behavior pred…

2024

Spatial-Logic-Aware Weakly Supervised Learning for Flood Mapping on Earth Imagery

AAAI 2024technical

Flood mapping on Earth imagery is crucial for disaster management, but its efficacy is hampered by the lack of high-quality training labels. Given high-resolution Earth imagery with coarse and noisy training labels, a base deep neural network model, and a spatial knowledge base with label constraint…

2024

TPGP: Temporal-Parametric Optimization with Deep Grasp Prior for Dexterous Motion Planning

ICRA 2024poster

Grasping motion planning aims to find a feasible grasping trajectory in the configuration space given an input target grasp. While optimizing grasp motion with two or three-fingered grippers has been well studied, the study on natural grasp motion planning with a dexterous hand remains a very challe…

Cited by 2SourceScholar
2024

Tuning Stable Rank Shrinkage: Aiming at the Overlooked Structural Risk in Fine-tuning

CVPR 2024poster

Existing fine-tuning methods for computer vision tasks primarily focus on re-weighting the knowledge learned from the source domain during pre-training. They aim to retain beneficial knowledge for the target domain while suppressing unfavorable knowledge. During the pre-training and fine-tuning stag…

2023

A Discriminative Multi-Channel Noise Feature Representation Model for Image Manipulation Localization

ICASSP 2023accepted

Noise feature modules are commonly used in image manipulation localization. However, different noise learning modules can only target limited tampering methods. In actual image tampering localization tasks, the tampering methods are unknown; it is difficult for a single noise feature module to match…

Cited by 0SourceScholar
2023

A Parallel Attention Mechanism for Image Manipulation Detection and Localization

ICASSP 2023accepted

Existing image manipulation detection and localization methods tend to detect the trail of manipulation and achieve decent performance. So far, however, there has been little discussion about the category imbalance. In this paper, we propose a parallel attention mechanism based network to localize t…

Cited by 0SourceScholar
2023

Contact2Grasp: 3D Grasp Synthesis via Hand-Object Contact Constraint

IJCAI 2023poster

3D grasp synthesis generates grasping poses given an input object. Existing works tackle the problem by learning a direct mapping from objects to the distributions of grasping poses. However, because the physical contact is sensitive to small changes in pose, the high-nonlinear mapping between 3D ob…

Cited by 15SourcePDFScholar
2023

ContactGen: Generative Contact Modeling for Grasp Generation

ICCV 2023poster

This paper presents a novel object-centric contact representation ContactGen for hand-object interaction. The ContactGen comprises 3 components: a contact map indicates the contact location, a part map represents the contact hand part, and a direction map tells the contact direction within each part…

Cited by 30PDFcodeScholar
2023

Diffuse3D: Wide-Angle 3D Photography via Bilateral Diffusion

ICCV 2023poster

This paper aims to resolve the challenging problem of wide-angle novel view synthesis from a single image, a.k.a. wide-angle 3D photography. Existing approaches rely on local context and treat them equally to inpaint occluded RGB and depth regions, which fail to deal with large-region occlusion (i.e…

Cited by 8PDFcodeScholar
2023

Dimension-independent Certified Neural Network Watermarks via Mollifier Smoothing

ICML 2023poster

Certified\_Watermarks is the first to provide a watermark certificate against $l_2$-norm watermark removal attacks, by leveraging the randomized smoothing techniques for certified robustness to adversarial attacks. However, the randomized smoothing techniques suffer from hardness of certified robust…

Cited by 6SourcePDFScholar
2023

Exploring the Effectiveness of Multi-Lingual Commonsense Knowledge-Aware Open-Domain Dialogue Response Generation

EMNLP 2023long findings

Prior works have shown the promising results of commonsense knowledge-aware models in improving informativeness while reducing the hallucination issue. Nonetheless, prior works often can only use monolingual knowledge whose language is consistent with the dialogue context. Except for a few high-reso…

Cited by 0SourceScholar
2023

Fast Federated Machine Unlearning with Nonlinear Functional Theory

ICML 2023poster

Federated machine unlearning (FMU) aims to remove the influence of a specified subset of training data upon request from a trained federated learning model. Despite achieving remarkable performance, existing FMU techniques suffer from inefficiency due to two sequential operations of training and ret…

Cited by 57SourcePDFScholar
2023

Federated Learning of Large Language Models with Parameter-Efficient Prompt Tuning and Adaptive Optimization

EMNLP 2023long main

Federated learning (FL) is a promising paradigm to enable collaborative model training with decentralized data. However, the training process of Large Language Models (LLMs) generally incurs the update of significant parameters, which limits the applicability of FL techniques to tackle the LLMs in r…

Cited by 0SourcecodeScholar
2023

Learning Navigational Visual Representations with Semantic Map Supervision

ICCV 2023poster

Being able to perceive the semantics and the spatial structure of the environment is essential for visual navigation of a household robot. However, most existing works only employ visual backbones pre-trained either with independent images for classification or with self-supervised learning methods…

Cited by 31PDFcodeScholar
2023

Tell2Design: A Dataset for Language-Guided Floor Plan Generation

ACL 2023long

We consider the task of generating designs directly from natural language descriptions, and consider floor plan generation as the initial research area. Language conditional generative models have recently been very successful in generating high-quality artistic images. However, designs must satisfy…

2022

APES: Articulated Part Extraction From Sprite Sheets

CVPR 2022poster

Rigged puppets are one of the most prevalent representations to create 2D character animations. Creating these puppets requires partitioning characters into independently moving parts. In this work, we present a method to automatically identify such articulated parts from a small set of character po…

Cited by 5PDFcodeScholar
2022

Accelerated Federated Learning with Decoupled Adaptive Optimization

ICML 2022spotlight

The federated learning (FL) framework enables edge clients to collaboratively learn a shared inference model while keeping privacy of training data on clients. Recently, many heuristics efforts have been made to generalize centralized adaptive optimization methods, such as SGDM, Adam, AdaGrad, etc.,…

Cited by 58SourcePDFScholar
2022

Audio-Driven Neural Gesture Reenactment With Video Motion Graphs

CVPR 2022poster

Human speech is often accompanied by body gestures including arm and hand gestures. We present a method that reenacts a high-quality video with gestures matching a target speech audio. The key idea of our method is to split and re-assemble clips from a reference video through a novel video motion gr…

Cited by 19PDFcodeScholar
2022

Diversity Matters: Fully Exploiting Depth Clues for Reliable Monocular 3D Object Detection

CVPR 2022oral

As an inherently ill-posed problem, depth estimation from single images is the most challenging part of monocular 3D object detection (M3OD). Many existing methods rely on preconceived assumptions to bridge the missing spatial information in monocular images, and predict a sole depth value for every…

Cited by 78PDFScholar
2022

Efficient Device Scheduling with Multi-Job Federated Learning

AAAI 2022technical

Recent years have witnessed a large amount of decentralized data in multiple (edge) devices of end-users, while the aggregation of the decentralized data remains difficult for machine learning jobs due to laws or regulations. Federated Learning (FL) emerges as an effective approach to handling decen…

Cited by 44SourcePDFScholar
2022

Faithful Extreme Rescaling via Generative Prior Reciprocated Invertible Representations

CVPR 2022oral

This paper presents a Generative prior ReciprocAted Invertible rescaling Network (GRAIN) for generating faithful high-resolution (HR) images from low-resolution (LR) invertible images with an extreme upscaling factor (64x). Previous researches have leveraged the prior knowledge of a pretrained GAN m…

Cited by 15PDFcodeScholar
2022

FedDUAP: Federated Learning with Dynamic Update and Adaptive Pruning Using Shared Data on the Server

IJCAI 2022poster

Despite achieving remarkable performance, Federated Learning (FL) suffers from two critical challenges, i.e., limited computational resources and low training efficiency. In this paper, we propose a novel FL framework, i.e., FedDUAP, with two original contributions, to exploit the insensitive data o…

Cited by 52SourcePDFScholar
2022

Input-agnostic Certified Group Fairness via Gaussian Parameter Smoothing

ICML 2022spotlight

Only recently, researchers attempt to provide classification algorithms with provable group fairness guarantees. Most of these algorithms suffer from harassment caused by the requirement that the training and deployment data follow the same distribution. This paper proposes an input-agnostic certifi…

2022

Learning Visibility for Robust Dense Human Body Estimation

ECCV 2022poster

"Estimating 3D human pose and shape from 2D images is a crucial yet challenging task. While prior methods with model-based representations can perform reasonably well on whole-body images, they often fail when parts of the body are occluded or outside the frame. Moreover, these results usually do no…

2022

Prompt Certified Machine Unlearning with Randomized Gradient Smoothing and Quantization

NeurIPS 2022accept

The right to be forgotten calls for efficient machine unlearning techniques that make trained machine learning models forget a cohort of data. The combination of training and unlearning operations in traditional machine unlearning methods often leads to the expensive computational cost on large-scal…

Cited by 39SourcePDFScholar
2022

Skeleton-Free Pose Transfer for Stylized 3D Characters

ECCV 2022poster

"We present the first method that automatically transfers poses between stylized 3D characters without skeletal rigging. In contrast to previous attempts to learn pose transformations on fixed or topology-equivalent skeleton templates, our method focuses on a novel scenario to handle skeleton-free c…

Cited by 41SourcePDFScholar
2022

Vision-based Relative Detection and Tracking for Teams of Micro Aerial Vehicles

IROS 2022poster

In this paper, we address the vision-based detection and tracking problems of multiple aerial vehicles using a single camera and Inertial Measurement Unit (IMU) as well as the corresponding perception consensus problem (i.e., uniqueness and identical IDs across all observing agents). We design sever…

Cited by 13SourceScholar
2021

Adversarial Attack against Cross-lingual Knowledge Graph Alignment

EMNLP 2021main

Recent literatures have shown that knowledge graph (KG) learning models are highly vulnerable to adversarial attacks. However, there is still a paucity of vulnerability analyses of cross-lingual entity alignment under adversarial attacks. This paper proposes an adversarial attack model with two nove…

Cited by 17SourcePDFScholar
2021

Expressive 1-Lipschitz Neural Networks for Robust Multiple Graph Learning against Adversarial Attacks

ICML 2021spotlight

Recent findings have shown multiple graph learning models, such as graph classification and graph matching, are highly vulnerable to adversarial attacks, i.e. small input perturbations in graph structures and node attributes can cause the model failures. Existing defense techniques often defend spec…

Cited by 31SourcePDFScholar
2021

Integrated Defense for Resilient Graph Matching

ICML 2021spotlight

A recent study has shown that graph matching models are vulnerable to adversarial manipulation of their input which is intended to cause a mismatching. Nevertheless, there is still a lack of a comprehensive solution for further enhancing the robustness of graph matching against adversarial attacks.…

Cited by 19SourcePDFScholar
2021

Knowledge-Aware Dialogue Generation via Hierarchical Infobox Accessing and Infobox-Dialogue Interaction Graph Network

IJCAI 2021poster

Due to limited knowledge carried by queries, traditional dialogue systems often face the dilemma of generating boring responses, leading to poor user experience. To alleviate this issue, this paper proposes a novel infobox knowledge-aware dialogue generation approach, HITA-Graph, with three unique f…

2021

More is Better: Enhancing Open-Domain Dialogue Generation via Multi-Source Heterogeneous Knowledge

EMNLP 2021main

Despite achieving remarkable performance, previous knowledge-enhanced works usually only use a single-source homogeneous knowledge base of limited knowledge coverage. Thus, they often degenerate into traditional methods because not all dialogues can be linked with knowledge entries. This paper propo…

2021

Validating the Lottery Ticket Hypothesis with Inertial Manifold Theory

NeurIPS 2021poster

Despite achieving remarkable efficiency, traditional network pruning techniques often follow manually-crafted heuristics to generate pruned sparse networks. Such heuristic pruning strategies are hard to guarantee that the pruned networks achieve test accuracy comparable to the original dense ones. R…

Cited by 37SourcePDFScholar
2021

What the Role is vs. What Plays the Role: Semi-Supervised Event Argument Extraction via Dual Question Answering

AAAI 2021technical

Event argument extraction is an essential task in event extraction, and become particularly challenging in the case of low-resource scenarios. We solve the issues in existing studies under low-resource situations from two sides. From the perspective of the model, the existing methods always suffer f…

2020

Adversarial Attacks on Deep Graph Matching

NeurIPS 2020poster

Despite achieving remarkable performance, deep graph learning models, such as node classification and network embedding, suffer from harassment caused by small adversarial perturbations. However, the vulnerability analysis of graph matching under adversarial attacks has not been fully investigated y…

Cited by 44SourcePDFScholar
2020

TopicKA: Generating Commonsense Knowledge-Aware Dialogue Responses Towards the Recommended Topic Fact

IJCAI 2020poster

Insufficient semantic understanding of dialogue always leads to the appearance of generic responses, in generative dialogue systems. Recently, high-quality knowledge bases have been introduced to enhance dialogue understanding, as well as to reduce the prevalence of boring responses. Although such k…

2019

Adaptive Vision-Based Control for Rope-Climbing Robot Manipulator

IROS 2019poster

While the mechanism of Rope-Climbing provides much flexibility, it opens up challenges to the development of the controller for Robotic Manipulator installed on Rope-Climbing robot(RCR), which is called Rope-Climbing Robot Manipulator(RCRM) here. In particular, the deformable nature of the rope resu…

Cited by 7SourceScholar
2019

ETNet: Error Transition Network for Arbitrary Style Transfer

NeurIPS 2019poster

Numerous valuable efforts have been devoted to achieving arbitrary style transfer since the seminal work of Gatys et al. However, existing state-of-the-art approaches often generate insufficiently stylized results under challenging cases. We believe a fundamental reason is that these approaches try…

2019

Global Vision-Based Impedance Control for Robotic Wall Polishing

IROS 2019poster

Wall polishing is a typical and essential procedure in the interior renovation. However, such works are mainly carried out by humans, which have the disadvantages of low efficiency, inconsistent quality, and issues of safety and health. A new vision-based impedance controller is proposed for polishi…

Cited by 18SourceScholar
2019

SceneGraphNet: Neural Message Passing for 3D Indoor Scene Augmentation

ICCV 2019poster

In this paper we propose a neural message passing approach to augment an input 3D indoor scene with new objects matching their surroundings. Given an input, potentially incomplete, 3D scene and a query location, our method predicts a probability distribution over object types that fit well in that l…

Cited by 106PDFcodeScholar
2018

A Synchronization Scheme for Position Control of Multiple Rope-Climbing Robots

ICRA 2018poster

The ability of rope-climbing robots in aloft operation is limited by its self-supporting and locomotion ability. In many applications, a given task is also too complex to be achieved by a single rope-climbing robot acting alone. The solution of multiple rope-climbing robots can overcome the limitati…

Cited by 5SourceScholar
2017

CSMA/CA-based electrocommunication system design for underwater robot groups

IROS 2017poster

Underwater communication is particularly challenging for small submarine robots that have limited power and size constraints. Inspired by weakly electric fish, a novel electric current communication (termed electrocommunication) system has been developed for small underwater robots in our previous s…

Cited by 17SourceScholar
2016

Cascaded Interactional Targeting Network for Egocentric Video Analysis

CVPR 2016poster

Knowing how hands move and what object is being manipulated are two key sub-tasks for analyzing first-person (egocentric) action. However, lack of fully annotated hand data as well as imprecise foreground segmentation make either sub-task challenging. This work aims to explicitly address these two i…

Cited by 68PDFScholar
2015

Interaction Part Mining: A Mid-Level Approach for Fine-Grained Action Recognition

CVPR 2015poster

Modeling human-object interactions and manipulating motions lies in the heart of fine-grained action recognition. Previous methods heavily rely on explicit detection of the object being interacted, which requires intensive human labour on object annotation. To bypass this constraint and achieve bett…

Cited by 103SourcePDFScholar
2015

Unsupervised Trajectory Clustering via Adaptive Multi-Kernel-Based Shrinkage

ICCV 2015poster

This paper proposes a shrinkage-based framework for unsupervised trajectory clustering. Facing to the challenges of trajectory clustering, e.g., large variations within a cluster and ambiguities across clusters, we first introduce an adaptive multi-kernel-based estimation process to estimate the `sh…

Cited by 96PDFScholar