← Search

Yujiu Yang

112 accepted papers

2026

AdaViewPlanner: Adapting Video Diffusion Models for Viewpoint Planning in 4D Scenes

ICLR 2026poster

Recent Text-to-Video (T2V) models have demonstrated powerful capability in visual simulation of real-world geometry and physical laws, indicating its potential as implicit world models. Inspired by this, we explore the feasibility of leveraging the video generation prior for viewpoint planning from…

Cited by 0SourceScholar
2026

FlowSteer: Guiding Few-Step Image Synthesis with Authentic Trajectories

CVPR 2026

With the success of flow matching in visual generation, sampling efficiency remains a critical bottleneck for its practical application. Among flow models' accelerating methods, ReFlow has been somehow overlooked although it has theoretical consistency with flow matching. This is primarily due to it

Cited by 0SourceScholar
2026

From Narrow to Panoramic Vision: Attention-Guided Cold-Start Reshapes Multimodal Reasoning

ICLR 2026poster

The cold-start initialization stage plays a pivotal role in training Multimodal Large Reasoning Models (MLRMs), yet its mechanisms remain insufficiently understood. To analyze this stage, we introduce the Visual Attention Score (VAS), an attention-based metric that quantifies how much a model attend…

Cited by 0SourcecodeScholar
2026

Generative Universal Verifier as Multimodal Meta-Reasoner

ICLR 2026oral

We introduce *Generative Universal Verifier*, a novel concept and plugin designed for next-generation multimodal reasoning in vision-language models and unified multimodal models, providing the fundamental capability of reflection and refinement on visual outcomes during the reasoning and generation…

Cited by 0SourcecodeScholar
2026

Multimodal Meta-Verifier with Explicit Structured Recalibration

ICML 2026poster

Visual outcomes are increasingly central to multimodal large language models, making reliable and fine-grained verification essential for scaling generalist foundation models. In this work, we investigate ***multimodal meta-verification***, which leverages verifier-generated rationales rather than d…

Cited by 0SourceScholar
2026

O-DisCo-Edit: Object Distortion Control for Unified Realistic Video Editing

AAAI 2026technical

Diffusion models have recently advanced video editing, yet controllable editing remains challenging due to the need for precise manipulation of diverse object properties. Current methods require different control signal for diverse editing tasks, which complicates model design and demands significan

Cited by 0SourcePDFScholar
2026

PixelCraft: A Multi-Agent system for High-Fidelity Visual Reasoning on Structured Images

ICLR 2026poster

Structured images (e.g., charts and geometric diagrams) remain challenging for multimodal large language models (MLLMs), as perceptual slips can cascade into erroneous conclusions. Intermediate visual cues can steer reasoning; however, existing cue-based methods are constrained with low-fidelity ima…

Cited by 0SourceScholar
2026

Reward Modeling from Natural Language Human Feedback

ICML 2026poster

Reinforcement Learning with Verifiable Reward (RLVR) on preference data has become the mainstream approach for training Generative Reward Models (GRMs). Typically, GRMs generate reasoning chains ending with critiques and preference labels, with RLVR using label correctness as the training reward. Ho…

Cited by 0SourceScholar
2026

See Less, See Right: Bi-directional Perceptual Shaping For Multimodal Reasoning

CVPR 2026

Large vision-language models (VLMs) often benefit from intermediate visual cues, either injected via external tools or generated as latent visual tokens during reasoning, but these mechanisms still overlook fine-grained visual evidence (e.g., polylines in charts), generalize poorly across domains, a

Cited by 0SourcecodeScholar
2026

Smaller Models are Natural Explorers for Policy-Level Diversity in GRPO

ICML 2026poster

We identify a new dimension for enhancing rollout diversity in Group Relative Policy Optimization (GRPO) for LLMs. While GRPO relies on diverse rollouts, prevailing strategies primarily increase diversity by injecting more token-level randomness, which may introduce step-wise noise and leads to inco…

Cited by 0SourceScholar
2026

VideoZoomer: Reinforcement-Learned Temporal Focusing for Long Video Reasoning

ICLR 2026poster

Multimodal Large Language Models (MLLMs) have achieved remarkable progress in vision-language tasks yet remain limited in long video understanding due to the limited context window. Consequently, prevailing approaches tend to rely on uniform frame sampling or static pre-selection, which might overlo…

Cited by 0SourceScholar
2026

Wan-Weaver: Interleaved Multi-modal Generation via Decoupled Training

CVPR 2026

Recent unified models have made unprecedented progress in both understanding and generation. However, while most of them accept multi-modal inputs, they typically produce only single-modality outputs. This challenge of producing interleaved content is mainly due to training data scarcity and the dif

Cited by 0SourceScholar
2025

Advancing Visual Large Language Model for Multi-granular Versatile Perception

ICCV 2025poster

Perception is a fundamental task in the field of computer vision, encompassing a diverse set of subtasks that can be systematically categorized into four distinct groups based on two dimensions: prediction type and instruction type. Notably, existing researches often focus solely on a limited subset…

2025

Chain-of-Reasoning: Towards Unified Mathematical Reasoning in Large Language Models via a Multi-Paradigm Perspective

ACL 2025long

Large Language Models (LLMs) have made notable progress in mathematical reasoning, yet they often rely on single-paradigm reasoning that limits their effectiveness across diverse tasks. In this paper, we introduce Chain-of-Reasoning (CoR), a novel unified framework that integrates multiple reasoning…

2025

ChartMimic: Evaluating LMM's Cross-Modal Reasoning Capability via Chart-to-Code Generation

ICLR 2025poster

We introduce a new benchmark, ChartMimic, aimed at assessing the visually-grounded code generation capabilities of large multimodal models (LMMs). ChartMimic utilizes information-intensive visual charts and textual instructions as inputs, requiring LMMs to generate the corresponding code for chart r…

2025

CoHD: A Counting-Aware Hierarchical Decoding Framework for Generalized Referring Expression Segmentation

ICCV 2025poster

The newly proposed Generalized Referring Expression Segmentation (GRES) amplifies the formulation of classic RES by involving complex multiple/non-target scenarios. Recent approaches address GRES by directly extending the well-adopted RES frameworks with object-existence identification. However, the…

Cited by 0SourcePDFScholar
2025

Critical Tokens Matter: Token-Level Contrastive Estimation Enhances LLM’s Reasoning Capability

ICML 2025poster

Mathematical reasoning tasks pose significant challenges for large language models (LLMs) because they require precise logical deduction and sequence analysis. In this work, we introduce the concept of critical tokens -- elements within reasoning trajectories that significantly influence incorrect o…

2025

DnLUT: Ultra-Efficient Color Image Denoising via Channel-Aware Lookup Tables

CVPR 2025poster

While deep neural networks have revolutionized image denoising capabilities, their deployment on edge devices remains challenging due to substantial computational and memory requirements. To this end, we present DnLUT, an ultra-efficient lookup table-based framework that achieves high-quality color…

2025

Edge-free but Structure-aware: Prototype-Guided Knowledge Distillation from GNNs to MLPs

COLING 2025main

Distilling high-accuracy Graph Neural Networks (GNNs) to low-latency multilayer perceptrons (MLPs) on graph tasks has become a hot research topic. However, conventional MLP learning relies almost exclusively on graph nodes and fails to effectively capture the graph structural information. Previous m…

Cited by 8SourcePDFScholar
2025

EpiCoder: Encompassing Diversity and Complexity in Code Generation

ICML 2025poster

Existing methods for code generation use code snippets as seed data, restricting the complexity and diversity of the synthesized data. In this paper, we introduce a novel feature tree-based synthesis framework, which revolves around hierarchical code features derived from high-level abstractions of…

Cited by 4SourcePDFScholar
2025

Fair Text-Attributed Graph Representation Learning

EMNLP 2025

Text-Attributed Graphs (TAGs), which integrate text and graph structures, have recently gained traction, especially in web applications. However, as a graph structure, TAG representation learning (TAGRL) naturally inherits issues from Graph Neural Networks (GNNs), such as fairness. Moreover, previou

Cited by 0SourcePDFScholar
2025

From Evasion to Concealment: Stealthy Knowledge Unlearning for LLMs

ACL 2025finding

LLM Unlearning plays a crucial role in removing sensitive information from language models to mitigate potential misuse. However, previous approaches often treat nonsensical responses or template-based refusals (e.g., “Sorry, I cannot answer.”) as the unlearning target, which can give the impression…

Cited by 0SourcePDFScholar
2025

HyperSeg: Hybrid Segmentation Assistant with Fine-grained Visual Perceiver

CVPR 2025poster

This paper aims to address universal segmentation for image and video perception with the strong reasoning ability empowered by Visual Large Language Models (VLLMs). Despite significant progress in current unified segmentation methods, limitations in adaptation to both image and video scenarios, as…

2025

IDA-VLM: Towards Movie Understanding via ID-Aware Large Vision-Language Model

ICLR 2025poster

The rapid advancement of Large Vision-Language models (LVLMs) has demonstrated a spectrum of emergent capabilities. Nevertheless, current models only focus on the visual content of a single scenario, while their ability to associate instances across different scenes has not yet been explored, which…

2025

IDOL: Instant Photorealistic 3D Human Creation from a Single Image

CVPR 2025poster

Creating a high-fidelity, animatable 3D full-body avatar from a single image is a challenging task due to the diverse appearance and poses of humans and the limited availability of high-quality training data. To achieve fast and high-quality human reconstruction, this work rethinks the task from the…

2025

Improving Video Generation with Human Feedback

NeurIPS 2025poster

Video generation has achieved significant advances through rectified flow techniques, but issues like unsmooth motion and misalignment between videos and prompts persist. In this work, we develop a systematic pipeline that harnesses human feedback to mitigate these problems and refine the video gene…

Cited by 0SourceScholar
2025

InstructSeg: Unifying Instructed Visual Segmentation with Multi-modal Large Language Models

ICCV 2025poster

Boosted by Multi-modal Large Language Models (MLLMs), text-guided universal segmentation models for the image and video domains have made rapid progress recently. However, these methods are often developed separately for specific domains, overlooking the similarities in task settings and solutions a…

2025

Invisible Entropy: Towards Safe and Efficient Low-Entropy LLM Watermarking

EMNLP 2025

Logit-based LLM watermarking traces and verifies AI-generated content by maintaining green and red token lists and increasing the likelihood of green tokens during generation. However, it struggles in low-entropy scenarios, where predictable outputs make green token selection difficult without disru

2025

IterComp: Iterative Composition-Aware Feedback Learning from Model Gallery for Text-to-Image Generation

ICLR 2025poster

Advanced diffusion models like Stable Diffusion 3, Omost, and FLUX have made notable strides in compositional text-to-image generation. However, these methods typically exhibit distinct strengths for compositional generation, with some excelling in handling attribute binding and others in spatial re…

2025

LLM2: Let Large Language Models Harness System 2 Reasoning

NAACL 2025short

Large language models (LLMs) have exhibited impressive capabilities across a myriad of tasks, yet they occasionally yield undesirable outputs. We posit that these limitations are rooted in the foundational autoregressive architecture of LLMs, which inherently lacks mechanisms for differentiating bet…

2025

MorphMark: Flexible Adaptive Watermarking for Large Language Models

ACL 2025long

Watermarking by altering token sampling probabilities based on red-green list is a promising method for tracing the origin of text generated by large language models (LLMs). However, existing watermark methods often struggle with a fundamental dilemma: improving watermark effectiveness (the detectab…

2025

PeRL: Permutation-Enhanced Reinforcement Learning for Interleaved Vision-Language Reasoning

NeurIPS 2025poster

Inspired by the impressive reasoning capabilities demonstrated by reinforcement learning approaches like DeepSeek-R1, recent emerging research has begun exploring the use of reinforcement learning (RL) to enhance vision-language models (VLMs) for multimodal reasoning tasks. However, most existing mu…

Cited by 0SourceScholar
2025

PocketSR: The Super-Resolution Expert in Your Pocket Mobiles

NeurIPS 2025poster

Real-world image super-resolution (RealSR) aims to enhance the visual quality of in-the-wild images, such as those captured by mobile phones. While existing methods leveraging large generative models demonstrate impressive results, the high computational cost and latency make them impractical for ed…

Cited by 0SourceScholar
2025

ProReflow: Progressive Reflow with Decomposed Velocity

CVPR 2025poster

Diffusion models have achieved significant progress in both image and video generation while still suffering from huge computation costs. As an effective solution, rectified flow aims to rectify the diffusion process of diffusion models into a straight line for few-step and even one-step generation.…

Cited by 1SourcePDFScholar
2025

Scalable Image Tokenization with Index Backpropagation Quantization

ICCV 2025poster

Existing vector quantization (VQ) methods struggle with scalability, largely attributed to the instability of the codebook that undergoes partial updates during training. The codebook is prone to collapse as utilization decreases, due to the progressively widening distribution gap between non-activa…

Cited by 0SourcePDFScholar
2025

ShifCon: Enhancing Non-Dominant Language Capabilities with a Shift-based Multilingual Contrastive Framework

ACL 2025long

Although fine-tuning Large Language Models (LLMs) with multilingual data can rapidly enhance the multilingual capabilities of LLMs, they still exhibit a performance gap between the dominant language (e.g., English) and non-dominant ones due to the imbalance of training data across languages. To furt…

2025

Teaching Your Models to Understand Code via Focal Preference Alignment

EMNLP 2025

Preference learning extends the performance of Code LLMs beyond traditional supervised fine-tuning by leveraging relative quality comparisons. In existing approaches, a set of n candidate solutions is evaluated based on test case success rates, with the candidate demonstrating a higher pass rate bei

2025

ToM: Leveraging Tree-oriented MapReduce for Long-Context Reasoning in Large Language Models

EMNLP 2025

Large Language Models (LLMs), constrained by limited context windows, often face significant performance degradation when reasoning over long contexts. To address this, Retrieval-Augmented Generation (RAG) retrieves and reasons over chunks but frequently sacrifices logical coherence due to its relia

Cited by 0SourcePDFScholar
2025

Unlocking Multimodal Mathematical Reasoning via Process Reward Model

NeurIPS 2025poster

Process Reward Models (PRMs) have shown promise in enhancing the mathematical reasoning capabilities of Large Language Models (LLMs) through Test-Time Scaling (TTS). However, their integration into multimodal reasoning remains largely unexplored. In this work, we take the first step toward unlocking…

Cited by 0SourceScholar
2025

Wan-Move: Motion-controllable Video Generation via Latent Trajectory Guidance

NeurIPS 2025poster

We present Wan-Move, a simple and scalable framework that brings motion control to video generative models. Existing motion-controllable methods typically suffer from coarse control granularity and limited scalability, leaving their outputs insufficient for practical use. We narrow this gap by achie…

Cited by 0SourceScholar
2024

A Thorough Examination of Decoding Methods in the Era of LLMs

EMNLP 2024main

Decoding methods play an indispensable role in converting language models from next-token predictors into practical task solvers. Prior research on decoding methods, primarily focusing on task-specific models, may not extend to the current era of general-purpose large language models (LLMs). Moreove…

2024

Accelerating Diffusion Models for Inverse Problems through Shortcut Sampling

IJCAI 2024poster

Diffusion models have recently demonstrated an impressive ability to address inverse problems in an unsupervised manner. While existing methods primarily focus on modifying the posterior sampling process, the potential of the forward process remains largely unexplored. In this work, we propose Short…

2024

Addressing Entity Translation Problem via Translation Difficulty and Context Diversity

ACL 2024findings

Neural machine translation (NMT) systems often produce inadequate translations for named entities. In this study, we conducted preliminary experiments to examine the factors affecting the translation accuracy of named entities, specifically focusing on their translation difficulty and context divers…

2024

AgentBoard: An Analytical Evaluation Board of Multi-turn LLM Agents

NeurIPS 2024oral

Evaluating large language models (LLMs) as general-purpose agents is essential for understanding their capabilities and facilitating their integration into practical applications. However, the evaluation process presents substantial challenges. A primary obstacle is the benchmarking of agent perform…

2024

Bridging the Gap: A Unified Video Comprehension Framework for Moment Retrieval and Highlight Detection

CVPR 2024poster

Video Moment Retrieval (MR) and Highlight Detection (HD) have attracted significant attention due to the growing demand for video analysis. Recent approaches treat MR and HD as similar video grounding problems and address them together with transformer-based architecture. However we observe that the…

2024

CRITIC: Large Language Models Can Self-Correct with Tool-Interactive Critiquing

ICLR 2024poster

Recent developments in large language models (LLMs) have been impressive. However, these models sometimes show inconsistencies and problematic behavior, such as hallucinating facts, generating flawed code, or creating offensive and toxic content. Unlike these models, humans typically utilize externa…

2024

CoSeR: Bridging Image and Language for Cognitive Super-Resolution

CVPR 2024poster

Existing super-resolution (SR) models primarily focus on restoring local texture details often neglecting the global semantic information within the scene. This oversight can lead to the omission of crucial semantic details or the introduction of inaccurate textures during the recovery process. In o…

2024

Connecting Large Language Models with Evolutionary Algorithms Yields Powerful Prompt Optimizers

ICLR 2024poster

Large Language Models (LLMs) excel in various tasks, but they rely on carefully crafted prompts that often demand substantial human effort. To automate this process, in this paper, we propose a novel framework for discrete prompt optimization, called EvoPrompt, which borrows the idea of evolutionary…

Cited by 0SourcePDFScholar
2024

CriticBench: Benchmarking LLMs for Critique-Correct Reasoning

ACL 2024findings

The ability of Large Language Models (LLMs) to critique and refine their reasoning is crucial for their application in evaluation, feedback provision, and self-improvement. This paper introduces CriticBench, a comprehensive benchmark designed to assess LLMs’ abilities to critique and rectify their r…

2024

Encouraging Divergent Thinking in Large Language Models through Multi-Agent Debate

EMNLP 2024main

Modern large language models (LLMs) like ChatGPT have shown remarkable performance on general language tasks but still struggle on complex reasoning tasks, which drives the research on cognitive behaviors of LLMs to explore human-like problem-solving strategies. Along this direction, one representat…

2024

Hint-Enhanced In-Context Learning Wakes Large Language Models Up For Knowledge-Intensive Tasks

ICASSP 2024accepted

In-context learning (ICL) ability has emerged with the increasing scale of large language models (LLMs), enabling them to learn input-label mappings from demonstrations and perform well on downstream tasks. However, under the standard ICL setting, LLMs may sometimes neglect query-related information…

Cited by 0SourceScholar
2024

HoLLMwood: Unleashing the Creativity of Large Language Models in Screenwriting via Role Playing

EMNLP 2024finding

Generative AI has demonstrated unprecedented creativity in the field of computer vision, yet such phenomena have not been observed in natural language processing. In particular, large language models (LLMs) can hardly produce written works at the level of human experts due to the extremely high comp…

Cited by 7SourcePDFScholar
2024

InsCL: A Data-efficient Continual Learning Paradigm for Fine-tuning Large Language Models with Instructions

NAACL 2024long

Instruction tuning effectively optimizes Large Language Models (LLMs) for downstream tasks. Due to the changing environment in real-life applications, LLMs necessitate continual task-specific adaptation without catastrophic forgetting. Considering the heavy computational cost, replay-based Continual…

Cited by 35SourcePDFScholar
2024

MLLMGuard: A Multi-dimensional Safety Evaluation Suite for Multimodal Large Language Models

NeurIPS 2024poster

Powered by remarkable advancements in Large Language Models (LLMs), Multimodal Large Language Models (MLLMs) demonstrate impressive capabilities in manifold tasks. However, the practical application scenarios of MLLMs are intricate, exposing them to potential malicious instructions and thereby posin…

2024

Mitigating Reversal Curse in Large Language Models via Semantic-aware Permutation Training

ACL 2024findings

While large language models (LLMs) have achieved impressive performance across diverse tasks, recent studies showcase that causal LLMs suffer from the “reversal curse”. It is a typical example that the model knows “A’s father is B”, but is unable to reason “B’s child is A”. This limitation poses a c…

2024

Not All Tokens Are What You Need for Pretraining

NeurIPS 2024oral

Previous language model pre-training methods have uniformly applied a next-token prediction loss to all training tokens. Challenging this norm, we posit that ''Not all tokens in a corpus are equally important for language model training''. Our initial analysis examines token-level training dynamics…

Cited by 9SourcePDFScholar
2024

PTD-SQL: Partitioning and Targeted Drilling with LLMs in Text-to-SQL

EMNLP 2024main

Large Language Models (LLMs) have emerged as powerful tools for Text-to-SQL tasks, exhibiting remarkable reasoning capabilities. Different from tasks such as math word problem and commonsense reasoning, SQL solutions have a relatively fixed pattern. This facilitates the investigation of whether LLMs…

2024

Prior Relational Schema Assists Effective Contrastive Learning for Inductive Knowledge Graph Completion

COLING 2024main

Knowledge Graph Completion (KGC) is a task aimed at uncovering the inherent relationships among known knowledge triplets in a Knowledge Graph (KG) and subsequently predicting missing links. Presently, there is a rising interest in inductive knowledge graph completion, where missing links may pertain…

2024

RealCompo: Balancing Realism and Compositionality Improves Text-to-Image Diffusion Models

NeurIPS 2024poster

Diffusion models have achieved remarkable advancements in text-to-image generation. However, existing models still have many difficulties when faced with multiple-object compositional generation. In this paper, we propose ***RealCompo***, a new *training-free* and *transferred-friendly* text-to-imag…

2024

Rolling Shutter Correction with Intermediate Distortion Flow Estimation

CVPR 2024poster

This paper proposes to correct the rolling shutter (RS) distorted images by estimating the distortion flow from the global shutter (GS) to RS directly. Existing methods usually perform correction using the undistortion flow from the RS to GS. They initially predict the flow from consecutive RS frame…

2024

SciAgent: Tool-augmented Language Models for Scientific Reasoning

EMNLP 2024main

Scientific reasoning poses an excessive challenge for even the most advanced Large Language Models (LLMs). To make this task more practical and solvable for LLMs, we introduce a new task setting named tool-augmented scientific reasoning. This setting supplements LLMs with scalable toolsets, and shif…

Cited by 28SourcePDFScholar
2024

Spurious Feature Diversification Improves Out-of-distribution Generalization

ICLR 2024poster

Generalization to out-of-distribution (OOD) data is a critical challenge in machine learning. Ensemble-based methods, like weight space ensembles that interpolate model parameters, have been shown to achieve superior OOD performance. However, the underlying mechanism for their effectiveness remains…

Cited by 31SourcePDFScholar
2024

TextBind: Multi-turn Interleaved Multimodal Instruction-following in the Wild

ACL 2024findings

Large language models with instruction-following abilities have revolutionized the field of artificial intelligence. These models show exceptional generalizability to tackle various real-world tasks through their natural language interfaces. However, their performance heavily relies on high-quality…

Cited by 17SourcePDFScholar
2024

ToRA: A Tool-Integrated Reasoning Agent for Mathematical Problem Solving

ICLR 2024poster

Large language models have made significant progress in various language tasks, yet they still struggle with complex mathematics. In this paper, we propose ToRA a series of Tool-integrated Reasoning Agents designed to solve challenging mathematical problems by seamlessly integrating natural language…

2024

ToolBeHonest: A Multi-level Hallucination Diagnostic Benchmark for Tool-Augmented Large Language Models

EMNLP 2024main

Tool-augmented large language models (LLMs) are rapidly being integrated into real-world applications. Due to the lack of benchmarks, the community has yet to fully understand the hallucination issues within these models. To address this challenge, we introduce a comprehensive diagnostic benchmark,…

2024

Unchosen Experts Can Contribute Too: Unleashing MoE Models’ Power by Self-Contrast

NeurIPS 2024poster

Mixture-of-Experts (MoE) has emerged as a prominent architecture for scaling model size while maintaining computational efficiency. In MoE, each token in the input sequence activates a different subset of experts determined by a routing mechanism. However, the unchosen experts in MoE models do not c…

2024

Universal Segmentation at Arbitrary Granularity with Language Instruction

CVPR 2024poster

This paper aims to achieve universal segmentation of arbitrary semantic level. Despite significant progress in recent years specialist segmentation approaches are limited to specific tasks and data distribution. Retraining a new model for adaptation to new scenarios or settings takes expensive compu…

Cited by 16SourcePDFScholar
2024

Weight-Inherited Distillation for Task-Agnostic BERT Compression

NAACL 2024findings

Knowledge Distillation (KD) is a predominant approach for BERT compression.Previous KD-based methods focus on designing extra alignment losses for the student model to mimic the behavior of the teacher model.These methods transfer the knowledge in an indirect way.In this paper, we propose a novel We…

2023

3D GAN Inversion With Facial Symmetry Prior

CVPR 2023poster

Recently, a surge of high-quality 3D-aware GANs have been proposed, which leverage the generative power of neural rendering. It is natural to associate 3D GANs with GAN inversion methods to project a real image into the generator's latent space, allowing free-view consistent synthesis and editing, r…

Cited by 45SourcePDFScholar
2023

A Two-Branch Network for Video Anomaly Detection with Spatio-Temporal Feature Learning

ICASSP 2023accepted

Video anomaly detection is very challenging, as most anomalies are rare and inconclusive. Previous weakly supervised learning approaches utilize the classifier trained with video-level labels to locate anomalous clips from the video. However, the anomalous clips often contain both anomalies and nume…

Cited by 0SourceScholar
2023

Assessor360: Multi-sequence Network for Blind Omnidirectional Image Quality Assessment

NeurIPS 2023poster

Blind Omnidirectional Image Quality Assessment (BOIQA) aims to objectively assess the human perceptual quality of omnidirectional images (ODIs) without relying on pristine-quality image information. It is becoming more significant with the increasing advancement of virtual reality (VR) technology. H…

2023

AutoConv: Automatically Generating Information-seeking Conversations with Large Language Models

ACL 2023short

Information-seeking conversation, which aims to help users gather information through conversation, has achieved great progress in recent years. However, the research is still stymied by the scarcity of training data. To alleviate this problem, we propose AutoConv for synthetic conversation generati…

2023

D2Match: Leveraging Deep Learning and Degeneracy for Subgraph Matching

ICML 2023poster

Subgraph matching is a fundamental building block for graph-based applications and is challenging due to its high-order combinatorial nature. Existing studies usually tackle it by combinatorial optimization or learning-based methods. However, they suffer from exponential computational costs or searc…

2023

Feature Expansion for Graph Neural Networks

ICML 2023poster

Graph neural networks aim to learn representations for graph-structured data and show impressive performance in node classification. Recently, many methods have studied the representations of GNNs from the perspective of optimization goals and spectral graph theory. However, the feature space that d…

2023

GLeaD: Improving GANs With a Generator-Leading Task

CVPR 2023poster

Generative adversarial network (GAN) is formulated as a two-player game between a generator (G) and a discriminator (D), where D is asked to differentiate whether an image comes from real data or is produced by G. Under such a formulation, D plays as the rule maker and hence tends to dominate the co…

2023

Global Knowledge Calibration for Fast Open-Vocabulary Segmentation

ICCV 2023poster

Recent advancements in pre-trained vision-language models, such as CLIP, have enabled the segmentation of arbitrary concepts solely from textual inputs, a process commonly referred to as open-vocabulary semantic segmentation (OVS). However, existing OVS techniques confront a fundamental challenge: t…

Cited by 55PDFScholar
2023

MAP: Multimodal Uncertainty-Aware Vision-Language Pre-Training Model

CVPR 2023poster

Multimodal semantic understanding often has to deal with uncertainty, which means the obtained messages tend to refer to multiple targets. Such uncertainty is problematic for our interpretation, including inter- and intra-modal uncertainty. Little effort has studied the modeling of this uncertainty,…

2023

NewsDialogues: Towards Proactive News Grounded Conversation

ACL 2023findings

Hot news is one of the most popular topics in daily conversations. However, news grounded conversation has long been stymied by the lack of well-designed task definition and scarce data. In this paper, we propose a novel task, Proactive News Grounded Conversation, in which a dialogue system can proa…

2023

Question Answering as Programming for Solving Time-Sensitive Questions

EMNLP 2023long main

Question answering plays a pivotal role in human daily life because it involves our acquisition of knowledge about the world. However, due to the dynamic and ever-changing nature of real-world facts, the answer can be completely different when the time constraint in the question changes. Recently, L…

Cited by 0SourcecodeScholar
2023

RIFormer: Keep Your Vision Backbone Effective but Removing Token Mixer

CVPR 2023poster

This paper studies how to keep a vision backbone effective while removing token mixers in its basic building blocks. Token mixers, as self-attention for vision transformers (ViTs), are intended to perform information communication between different spatial tokens but suffer from considerable computa…

Cited by 37SourcePDFScholar
2023

Recouple Event Field via Probabilistic Bias for Event Extraction

ICASSP 2023accepted

Event Extraction (EE), aiming to identify and classify event triggers and arguments from event mentions, has benefited from pre-trained language models (PLMs). However, existing PLM-based methods ignore the information of trigger/argument fields, which is crucial for understanding event schemas. To…

Cited by 0SourceScholar
2023

SOC: Semantic-Assisted Object Cluster for Referring Video Object Segmentation

NeurIPS 2023poster

This paper studies referring video object segmentation (RVOS) by boosting video-level visual-linguistic alignment. Recent approaches model the RVOS task as a sequence prediction problem and perform multi-modal interaction as well as segmentation for each frame separately. However, the lack of a glob…

2023

Seeing What You Miss: Vision-Language Pre-Training With Semantic Completion Learning

CVPR 2023poster

Cross-modal alignment is essential for vision-language pre-training (VLP) models to learn the correct corresponding information across different modalities. For this purpose, inspired by the success of masked language modeling (MLM) tasks in the NLP pre-training area, numerous masked modeling tasks…

2023

Solving Math Word Problems via Cooperative Reasoning induced Language Models

ACL 2023long

Large-scale pre-trained language models (PLMs) bring new opportunities to challenging problems, especially those that need high-level intelligence, such as the math word problem (MWPs). However, directly applying existing PLMs to MWPs can fail as the generation process lacks sufficient supervision a…

2023

Specialist or Generalist? Instruction Tuning for Specific NLP Tasks

EMNLP 2023long main

The potential of large language models (LLMs) to simultaneously perform a wide range of natural language processing (NLP) tasks has been the subject of extensive research. Although instruction tuning has proven to be a data-efficient method for transforming LLMs into such generalist models, their pe…

Cited by 0SourceScholar
2023

Syngen: A Syntactic Plug-And-Play Module for Generative Aspect-Based Sentiment Analysis

ICASSP 2023accepted

Aspect-based Sentiment Analysis (ABSA) is a sentiment analysis task at fine-grained level. Recently, generative frameworks have attracted increasing attention in ABSA due to their ability to unify subtasks and their continuity to upstream pre-training tasks. However, these generative models suffer f…

Cited by 0SourceScholar
2023

UniKD: Universal Knowledge Distillation for Mimicking Homogeneous or Heterogeneous Object Detectors

ICCV 2023poster

Knowledge distillation (KD) has become a standard method to boost the performance of lightweight object detectors. Most previous works are feature-based, where students mimic the features of homogeneous teacher detectors. However, distilling the knowledge from the heterogeneous teacher fails in this…

Cited by 9PDFScholar
2022

Attention Probe: Vision Transformer Distillation in the Wild

ICASSP 2022accepted

Vision transformers (ViTs) require intensive computational resources to achieve high performance, which usually makes them not suitable for mobile devices. A feasible strategy is to compress them using the original training data, which may be not accessible due to privacy limitations or transmission…

Cited by 0SourceScholar
2022

Global Spectral Filter Memory Network for Video Object Segmentation

ECCV 2022poster

"This paper studies semi-supervised video object segmentation through boosting intra-frame interaction. Recent memory network-based methods focus on exploiting inter-frame temporal reference while paying little attention to intra-frame spatial dependency. Specifically, these segmentation model tends…

2022

High-Fidelity GAN Inversion with Padding Space

ECCV 2022poster

"Inverting a Generative Adversarial Network (GAN) facilitates a wide range of image editing tasks using pre-trained generators. Existing methods typically employ the latent space of GANs as the inversion space yet observe the insufficient recovery of spatial details. In this work, we propose to invo…

2022

Learning Adaptive Warping for Real-World Rolling Shutter Correction

CVPR 2022poster

This paper proposes a real-world rolling shutter (RS) correction dataset, BS-RSC, and a corresponding model to correct the RS frames in a distorted video. Mobile devices in the consumer market with CMOS-based sensors for video capture often result in rolling shutter effects when relative movements o…

Cited by 27PDFcodeScholar
2022

Learning Quality-Aware Dynamic Memory for Video Object Segmentation

ECCV 2022poster

"Recently, several spatial-temporal memory-based methods have verified that storing intermediate frames and their masks as memory are helpful to segment target objects in videos. However, they mainly focus on better matching between the current frame and the memory frames without explicitly paying a…

2022

Rethinking Alignment in Video Super-Resolution Transformers

NeurIPS 2022accept

The alignment of adjacent frames is considered an essential operation in video super-resolution (VSR). Advanced VSR models, including the latest VSR Transformers, are generally equipped with well-designed alignment modules. However, the progress of the self-attention mechanism may violate this commo…

2022

StyleHEAT: One-Shot High-Resolution Editable Talking Face Generation via Pre-trained StyleGAN

ECCV 2022poster

"One-shot talking face generation aims at synthesizing a high-quality talking face video from an arbitrary portrait image, driven by a video or an audio segment. In this work, we provide a solution from a novel perspective that differs from existing frameworks. We first investigate the latent featur…

2021

MIRTT: Learning Multimodal Interaction Representations from Trilinear Transformers for Visual Question Answering

EMNLP 2021finding

In Visual Question Answering (VQA), existing bilinear methods focus on the interaction between images and questions. As a result, the answers are either spliced into the questions or utilized as labels only for classification. On the other hand, trilinear models such as the CTI model efficiently uti…

2021

More: A Metric Learning Based Framework for Open-Domain Relation Extraction

ICASSP 2021accepted

Open relation extraction (OpenRE) is the task of extracting relation schemes from open-domain corpora. Most existing OpenRE methods either do not fully benefit from high-quality labeled corpora or can not learn semantic representation directly, affecting downstream clustering efficiency. To address…

Cited by 0SourceScholar
2021

Probabilistic Modeling of Semantic Ambiguity for Scene Graph Generation

CVPR 2021poster

To generate "accurate" scene graphs, almost all exist-ing methods predict pairwise relationships in a determin-istic manner. However, we argue that visual relationshipsare often semantically ambiguous. Specifically, inspired bylinguistic knowledge, we classify the ambiguity into threetypes: Synonymy…

Cited by 81PDFcodeScholar
2021

TediGAN: Text-Guided Diverse Face Image Generation and Manipulation

CVPR 2021poster

In this work, we propose TediGAN, a novel framework for multi-modal image generation and manipulation with textual descriptions. The proposed method consists of three components: StyleGAN inversion module, visual-linguistic similarity learning, and instance-level optimization. The inversion module m…

Cited by 472PDFcodeScholar
2020

DT-QDC: A Dataset for Question Comprehension in Online Test

COLING 2020main

With the transformation of education from the traditional classroom environment to online education and assessment, it is more and more important to accurately assess the difficulty of questions than ever. As teachers may not be able to follow the student’s performance and learning behavior closely,…

2020

Sparse Adversarial Attack via Perturbation Factorization

ECCV 2020poster

This work studies the sparse adversarial attack, which aims to generate adversarial perturbations onto partial positions of one benign image, such that the perturbed image is incorrectly predicted by one deep neural network (DNN) model. The sparse adversarial attack involves two challenges, i.e., wh…

2019

Compressing Convolutional Neural Networks via Factorized Convolutional Filters

CVPR 2019poster

This work studies the model compression for deep convolutional neural networks (CNNs) via filter pruning. The workflow of a traditional pruning consists of three sequential stages: pre-training the original model, selecting the pre-trained filters via ranking according to a manually designed criteri…

Cited by 133PDFcodeScholar