← Search

Bo Zheng

110 accepted papers

2026

A Semantically Consistent Dataset for Data-Efficient Query-Based Universal Sound Separation

ICML 2026poster

Query-based universal sound separation is fundamental to intelligent auditory systems, aiming to isolate specific sources from mixtures. Despite recent advances, existing methods continue to suffer from residual interference in complex acoustic scenes. This performance limitation stems largely from …

Cited by 0SourcecodeScholar
2026

AdaSpark: Adaptive Sparsity for Efficient Long-Video Understanding

CVPR 2026

Processing long-form videos with Video Large Language Models (Video-LLMs) is computationally prohibitive. Current efficiency methods often compromise fine-grained perception through irreversible information disposal or inhibit long-range temporal modeling via rigid, predefined sparse patterns. This

Cited by 0SourceScholar
2026

Asymmetric Proximal Policy Optimization: mini-critics boost LLM reasoning

ICLR 2026poster

Reinforcement learning (RL) has become a central paradigm for post-training large language models (LLMs) to elicit stronger reasoning. Yet, most recent RL for LLMs (RL4LLM) methods avoid explicit critics, replacing them with average advantage baselines. This shift is largely pragmatic: conventional…

Cited by 0SourceScholar
2026

Attention Illuminates LLM Reasoning: The Uncovered Preplan-and-Anchor Rhythm Enables Fine-Grained Policy Optimization

ICML 2026poster

The reasoning patterns of large language models (LLMs) remain opaque, and Reinforcement learning (RL) typically assigns uniform credit across an entire generation, blurring the distinction between pivotal and routine steps. This work treats attention as a natural substrate for interpreting LLM reaso…

Cited by 0SourceScholar
2026

COMI: Coarse-to-fine Context Compression via Marginal Information Gain

ICLR 2026poster

Large Language Models (LLMs) have demonstrated exceptional capabilities across diverse tasks. However, their deployment in long context scenarios remains hindered by computational inefficiency and information redundancy. Context compression methods address these challenges by significantly reducing…

Cited by 0SourcecodeScholar
2026

Contribution-aware Token Compression for Efficient Video Understanding via Reinforcement Learning

AAAI 2026technical

Video large language models have demonstrated remarkable capabilities in video understanding tasks. However, the redundancy of video tokens introduces significant computational overhead during inference, limiting their practical deployment. Many compression algorithms are proposed to prioritize reta

Cited by 0SourcePDFScholar
2026

Cross-modal Identity Mapping: Minimizing Information Loss in Modality Conversion via Reinforcement Learning

CVPR 2026

Large Vision-Language Models (LVLMs) often omit or misrepresent critical visual content in generated image captions. Minimizing such information loss will force LVLMs to focus on image details to generate precise descriptions. However, measuring information loss during modality conversion is inheren

Cited by 0SourceScholar
2026

DESIGNER: Design-Logic-Guided Multidisciplinary Data Synthesis for LLM Reasoning

ICLR 2026poster

Large language models (LLMs) perform strongly on many language tasks but still struggle with complex multi-step reasoning across disciplines. Existing reasoning datasets often lack disciplinary breadth, reasoning depth, and diversity, as well as guiding principles for question synthesis. We propose…

Cited by 0SourceScholar
2026

DeepPhy: Benchmarking Agentic VLMs on Physical Reasoning

AAAI 2026technical

Although Vision Language Models (VLMs) exhibit strong perceptual abilities and impressive visual reasoning, they struggle with attention to detail and precise action planning in complex, dynamic environments, leading to subpar performance. Real-world tasks typically require complex interactions, adv

Cited by 0SourcePDFScholar
2026

E-VAds: An E-commerce Short Videos Understanding Benchmark for MLLMs

ICML 2026poster

E-commerce short videos represent a high-revenue segment of the online video industry characterized by a goal-driven format and dense multi-modal signals. Current models often struggle with these videos because existing benchmarks focus primarily on general-purpose tasks and neglect the reasoning of…

Cited by 1SourceScholar
2026

Enhancing Generative Auto-bidding with Offline Reward Evaluation and Policy Search

ICLR 2026oral

Auto-bidding serves as a critical tool for advertisers to improve their advertising performance. Recent progress has demonstrated that AI-Generated Bidding (AIGB), which learns a conditional generative planner from offline data, achieves superior performance compared to typical offline reinforcement…

Cited by 0SourceScholar
2026

Expert Divergence Learning for MoE-based Language Models

ICLR 2026poster

The Mixture-of-Experts (MoE) architecture is a powerful technique for scaling language models, yet it often suffers from expert homogenization, where experts learn redundant functionalities, thereby limiting MoE's full potential. To address this, we introduce Expert Divergence Learning, a novel pre-…

Cited by 0SourceScholar
2026

FloodDiffusion: Tailored Diffusion Forcing for Streaming Motion Generation

CVPR 2026

We present FloodDiffusion, a new framework for text-driven, streaming human motion generation. Given time-varying text prompts, FloodDiffusion generates text-aligned, seamless motion sequences with real-time latency.Unlike existing methods that rely on chunk-by-chunk or auto-regressive model with di

Cited by 0SourceScholar
2026

GIFT: Global Irreplaceability Frame Targeting for Efficient Video Understanding

CVPR 2026

Video Large Language Models (VLMs) have achieved remarkable success in video understanding, but the significant computational cost from processing dense frames severely limits their practical application. Existing methods alleviate this by selecting keyframes, but their greedy decision-making, combi

Cited by 0SourceScholar
2026

Graph2Eval: Automatic Multimodal Task Generation for Agents via Knowledge Graphs

CVPR 2026

As multimodal LLM-driven agents advance in autonomy and generalization, traditional static datasets face inherent scalability limitations and are insufficient for fully assessing their capabilities in increasingly complex and diverse tasks. Existing studies have attempted to generate agent tasks usi

Cited by 0SourcecodeScholar
2026

HiDe: Rethinking The Zoom-IN method in High Resolution MLLMs via Hierarchical Decoupling

ICML 2026poster

Multimodal Large Language Models (MLLMs) have made substantial progress on visual understanding tasks, yet they still perform poorly on high-resolution images. Prior work often attributes this limitation to perceptual constraints, arguing that MLLMs fail to recognize small objects and therefore rely…

Cited by 0SourceScholar
2026

How Does Reasoning Flow? Tracing Attention-Induced Information Flow for Targeted RL in LLMs

ICML 2026poster

Token-level credit assignment remains a key obstacle for reinforcement learning (RL) in large language models (LLMs), where RL recipes typically treat all tokens equally, failing to distinguish decisive reasoning steps from routine formatting or fluent filler. Recent attempts leverage model-internal…

Cited by 0SourceScholar
2026

Identity-Preserving Image-to-Video Generation via Reward-Guided Optimization

CVPR 2026

Recent advances in image-to-video (I2V) generation have achieved remarkable progress in synthesizing high-quality, temporally coherent videos from static images. Among all the applications of I2V, human-centric video generation includes a large portion. However, existing I2V models encounter difficu

Cited by 0SourcecodeScholar
2026

MIRAGE: Towards AI-Generated Image Detection in the Wild

AAAI 2026technical

The spreading of AI-generated images (AIGI), driven by advances in generative AI, poses a significant threat to in- formation security and public trust. Existing AIGI detectors, while effective against images in clean laboratory settings, fail to generalize to in-the-wild scenarios. These real-world

Cited by 0SourcePDFScholar
2026

MOON2.0: Dynamic Modality-balanced Multimodal Representation Learning for E-commerce Product Understanding

CVPR 2026

Recent Multimodal Large Language Models (MLLMs) have significantly advanced e-commerce product understanding. However, they still face three challenges: (i) the modality imbalance induced by modality mixed training; (ii) underutilization of the intrinsic alignment relationships among visual and text

Cited by 0SourceScholar
2026

MeSH: Memory-as-State-Highways for Recursive Transformers

ICLR 2026poster

Recursive transformers reuse parameters and iterate over hidden states multiple times, decoupling compute depth from parameter depth. However, under matched compute, recursive models with fewer parameters often lag behind non-recursive counterparts. By probing hidden states, we trace this performanc…

Cited by 0SourcecodeScholar
2026

Mixture-of-Visual-Thoughts: Exploring Context-Adaptive Reasoning Mode Selection for General Visual Reasoning

ICLR 2026poster

Current visual reasoning methods mainly focus on exploring specific reasoning modes. Although improvements can be achieved in particular domains, they struggle to develop general reasoning capabilities. Inspired by this, we propose a novel adaptive reasoning paradigm, $\underline{\text{M}}$ixture-$\…

Cited by 0SourcecodeScholar
2026

ORION: Decoupling and Alignment for Unified Autoregressive Understanding and Generation

ICLR 2026poster

Unified multimodal Large Language Models (MLLMs) hold great promise for seamlessly integrating understanding and generation. However, monolithic autoregressive architectures, despite their elegance and conversational fluency, suffer from a fundamental semantic–structural conflict: optimizing for low…

Cited by 0SourceScholar
2026

ReWatch-R1: Boosting Complex Video Reasoning in Large Vision-Language Models through Agentic Data Synthesis

ICLR 2026poster

While Reinforcement Learning with Verifiable Reward (RLVR) significantly advances image reasoning in Large Vision-Language Models (LVLMs), its application to complex video reasoning remains underdeveloped. This gap stems primarily from a critical data bottleneck: existing datasets lack the challengi…

Cited by 0SourcecodeScholar
2026

Reasoning Palette: Modulating Reasoning via Latent Contextualization for Controllable Exploration for (V)LMs

CVPR 2026

Exploration capacity shapes both inference-time performance and reinforcement learning (RL) training for large (vision-) language models, as stochastic sampling often yields redundant reasoning paths with little high-level diversity. This paper proposes Reasoning Palette, a novel latent-modulation f

Cited by 0SourceScholar
2026

Reconstructing KV Caches with Cross-Layer Fusion for Enhanced Transformers

ICLR 2026poster

Transformer decoders have achieved strong results across tasks, but the memory required for the KV cache becomes prohibitive at long sequence lengths. Although Cross-layer KV Cache sharing (e.g., YOCO, CLA) offers a path to mitigate KV Cache bottleneck, it typically underperforms within-layer method…

Cited by 0SourcecodeScholar
2026

RuCL: Stratified Rubric-Based Curriculum Learning for Multimodal Large Language Model Reasoning

ICML 2026poster

Reinforcement Learning with Verifiable Rewards (RLVR) has emerged as a prevailing paradigm for enhancing reasoning in Multimodal Large Language Models (MLLMs). However, relying solely on outcome supervision risks reward hacking, where models learn spurious reasoning patterns to satisfy final answer …

Cited by 4SourceScholar
2026

SpatialReward: Verifiable Spatial Reward Modeling for Fine-Grained Spatial Consistency in Text-to-Image Generation

CVPR 2026

Recent advances in text-to-image (T2I) generation via reinforcement learning (RL) have benefited from reward models that assess semantic alignment and visual quality. However, most existing reward models pay limited attention to fine-grained spatial relationships, often producing images that appear

Cited by 0SourcecodeScholar
2026

Stop Unnecessary Reflection: Training LRMs for Efficient Reasoning with Adaptive Reflection and Length Coordinated Penalty

ICLR 2026poster

Large Reasoning Models (LRMs) have demonstrated remarkable performance on complex reasoning tasks by employing test-time scaling. However, they often generate over-long chains-of-thought that, driven by substantial reflections such as repetitive self-questioning and circular reasoning, lead to high…

Cited by 0SourcecodeScholar
2026

The Flexibility Trap: Rethinking the Value of Arbitrary Order in Diffusion Language Models

ICML 2026oral

Diffusion Large Language Models (dLLMs) break the rigid left-to-right constraint of traditional LLMs, enabling token generation in arbitrary orders. Intuitively, this flexibility implies a solution space that strictly supersets the fixed autoregressive trajectory, theoretically unlocking superior re…

Cited by 0SourceScholar
2026

Thinking with Drafts: Speculative Temporal Reasoning for Efficient Long Video Understanding

CVPR 2026

Long video understanding is essential for human-like intelligence, enabling coherent perception and reasoning over extended temporal contexts. While the emerging thinking-with-frames paradigm--which alternates between global temporal reasoning and local frame examination--has advanced the reasoning

Cited by 0SourceScholar
2026

Tricks or Traps? A Deep Dive into RL for LLM Reasoning

ICLR 2026poster

Reinforcement learning (RL) for LLM reasoning has rapidly emerged as a prominent research area, marked by a significant surge in related studies on both algorithmic innovations and practical applications. Despite this progress, several critical challenges remain, including the absence of standardize…

Cited by 0SourcecodeScholar
2026

Trimming the Fat: Redundancy-Aware Acceleration Framework for DGNNs

AAAI 2026technical

Temporal graphs are essential for modeling complex real-world systems, such as social interactions, financial transactions, and recommendation systems, but the high computational cost and model complexity of dynamic graph neural networks (DGNNs) pose significant challenges for practical deployment.

Cited by 0SourcePDFScholar
2026

UniLS: End-to-End Audio-Driven Avatars for Unified Listening and Speaking

CVPR 2026

Generating lifelike conversational avatars requires modeling not just isolated speakers, but the dynamic, reciprocal interaction of speaking and listening.However, modeling the listener is exceptionally challenging: direct audio-driven training fails, producing stiff, static listening motions. This

Cited by 0SourcecodeScholar
2026

VSTYLE: A BENCHMARK FOR VOICE STYLE ADAPTATION WITH SPOKEN INSTRUCTIONS

ICASSP 2026poster

Spoken language models (SLMs) have emerged as a unified paradigm for speech understanding and generation, enabling natural human machine interaction. However, while most progress has focused on semantic accuracy and instruction following, the ability of SLMs to adapt their speaking style based on sp…

Cited by 0SourcePDFScholar
2026

iTryOn: Mastering Interactive Video Virtual Try-On with Spatial-Semantic Guidance

ICML 2026poster

Video Virtual Try-On (VVT) aims to seamlessly replace a garment on a person in a video with a new one. While existing methods have made significant strides in maintaining temporal consistency, they are predominantly confined to non-interactive scenarios where models merely showcase garments. This li…

Cited by 0SourceScholar
2025

2D-DPO: Scaling Direct Preference Optimization with 2-Dimensional Supervision

NAACL 2025findings

Recent advancements in Direct Preference Optimization (DPO) have significantly enhanced the alignment of Large Language Models (LLMs) with human preferences, owing to its simplicity and effectiveness. However, existing methods typically optimize a scalar score or ranking reward, thereby overlooking…

Cited by 2SourcePDFScholar
2025

AIGuard: A Benchmark and Lightweight Detection for E-commerce AIGC Risks

ACL 2025finding

Recent advancements in AI-generated content (AIGC) have heightened concerns about harmful outputs, such as misinformation and malicious misuse.Existing detection methods face two key limitations:(1) lacking real-world AIGC scenarios and corresponding risk datasets, and(2) both traditional and multim…

2025

AIR: Complex Instruction Generation via Automatic Iterative Refinement

EMNLP 2025

With the development of large language models, their ability to follow simple instructions has significantly improved. However, adhering to complex instructions remains a major challenge. Current approaches to generating complex instructions are often irrelevant to the current instruction requiremen

2025

Can Large Language Models Detect Errors in Long Chain-of-Thought Reasoning?

ACL 2025long

Recently, o1-like models have drawn significant attention, where these models produce the long Chain-of-Thought (CoT) reasoning steps to improve the reasoning abilities of existing Large Language Models (LLMs). In this paper, to understand the qualities of these long CoTs and measure the critique ab…

2025

Chinese SafetyQA: A Safety Short-form Factuality Benchmark for Large Language Models

ACL 2025long

With the rapid advancement of Large Language Models (LLMs), significant safety concerns have emerged. Fundamentally, the safety of large language models is closely linked to the accuracy, comprehensiveness, and clarity of their understanding of safety knowledge, particularly in domains such as law,…

Cited by 0SourcePDFScholar
2025

Chinese SimpleQA: A Chinese Factuality Evaluation for Large Language Models

ACL 2025long

New LLM benchmarks are important to align with the rapid development of Large Language Models (LLMs). In this work, we present Chinese SimpleQA, the first comprehensive Chinese benchmark to evaluate the factuality ability of LLMs to answer short questions, and Chinese SimpleQA mainly has five proper…

2025

CombatVLA: An Efficient Vision-Language-Action Model for Combat Tasks in 3D Action Role-Playing Games

ICCV 2025poster

Recent advances in Vision-Language-Action models (VLAs) have expanded the capabilities of embodied intelligence. However, significant challenges remain in real-time decision-making in complex 3D environments, which demand second-level responses, high-resolution perception, and tactical reasoning und…

2025

D3: Diversity, Difficulty, and Dependability-Aware Data Selection for Sample-Efficient LLM Instruction Tuning

IJCAI 2025

Recent advancements in instruction tuning for large language models (LLMs) suggest that a small, high-quality dataset can significantly equip LLMs with instruction-following capabilities, outperforming large datasets often burdened by quality and redundancy issues. However, the challenge lies in aut

Cited by 0SourcePDFScholar
2025

Demons in the Detail: On Implementing Load Balancing Loss for Training Specialized Mixture-of-Expert Models

ACL 2025long

This paper revisits the implementation of Load-Balancing-Loss (LBL) when training Mixture-of-Experts (MoEs) models. Specifically, LBL for MoEs is defined as NE ∑i=1NE fipi, where NE is the total number of experts, fi represents the frequency of expert i being selected, and pi denotes the average gat…

2025

Differentiable Solver Search for Fast Diffusion Sampling

ICML 2025poster

Diffusion models have demonstrated remarkable generation quality but at the cost of numerous function evaluations. Recently, advanced ODE-based solvers have been developed to mitigate the substantial computational demands of reverse-diffusion solving under limited sampling steps. However, these solv…

Cited by 0SourcePDFScholar
2025

Do not Abstain! Identify and Solve the Uncertainty

ACL 2025long

Despite the widespread application of Large Language Models (LLMs) across various domains, they frequently exhibit overconfidence when encountering uncertain scenarios, yet existing solutions primarily rely on evasive responses (e.g., “I don’t know”) overlooks the opportunity of identifying and addr…

Cited by 0SourcePDFScholar
2025

Enhancing Document Understanding with Group Position Embedding: A Novel Approach to Incorporate Layout Information

ICLR 2025poster

Recent advancements in document understanding have been dominated by leveraging large language models (LLMs) and multimodal large models. However, enabling LLMs to comprehend complex document layouts and structural information often necessitates intricate network modifications or costly pre-training…

2025

Gated Attention for Large Language Models: Non-linearity, Sparsity, and Attention-Sink-Free

NeurIPS 2025oral

Gating mechanisms have been widely utilized, from early models like LSTMs and Highway Networks to recent state space models, linear attention, and also softmax attention. Yet, existing literature rarely examines the specific effects of gating. In this work, we conduct comprehensive experiments to sy…

Cited by 0SourcecodeScholar
2025

HiddenDetect: Detecting Jailbreak Attacks against Multimodal Large Language Models via Monitoring Hidden States

ACL 2025long

The integration of additional modalities increases the susceptibility of large vision-language models (LVLMs) to safety risks, such as jailbreak attacks, compared to their language-only counterparts. While existing research primarily focuses on post-hoc alignment techniques, the underlying safety me…

Cited by 0SourcePDFScholar
2025

How to inject knowledge efficiently? Knowledge Infusion Scaling Law for Pre-training Large Language Models

EMNLP 2025

Large language models (LLMs) have attracted significant attention due to their impressive general capabilities across diverse downstream tasks. However, without domain-specific optimization, they often underperform on specialized knowledge benchmarks and even produce hallucination. Recent studies sh

Cited by 0SourcePDFScholar
2025

INTER: Mitigating Hallucination in Large Vision-Language Models by Interaction Guidance Sampling

ICCV 2025poster

Hallucinations in large vision-language models (LVLMs) pose significant challenges for real-world applications, as LVLMs may generate responses that appear plausible yet remain inconsistent with the associated visual content. This issue rarely occurs in human cognition. We argue that this discrepanc…

2025

LLGS: Illuminating Gaussian Splatting via absorptance Modulation

ICASSP 2025accepted

Low-light images are typically characterized by low pixel intensity and color distortion, presenting a significant challenge for accurate 3D reconstruction with 3D Gaussian Splatting (3DGS). Traditional 2D enhancement methods fail to maintain consistent illumination, affecting reconstruction quality…

Cited by 0SourceScholar
2025

LeTS: Learning to Think-and-Search via Process-and-Outcome Reward Hybridization

EMNLP 2025

Large language models (LLMs) have demonstrated impressive capabilities in reasoning with the emergence of reasoning models like OpenAI-o1 and DeepSeek-R1. Recent research focuses on integrating reasoning capabilities into the realm of retrieval-augmented generation (RAG) via outcome-supervised reinf

2025

LongDocURL: a Comprehensive Multimodal Long Document Benchmark Integrating Understanding, Reasoning, and Locating

ACL 2025long

Large vision language models (LVLMs) have improved the document understanding capabilities remarkably, enabling the handling of complex document elements, longer contexts, and a wider range of tasks. However, existing document understanding benchmarks have been limited to handling only a small numbe…

2025

M2RC-EVAL: Massively Multilingual Repository-level Code Completion Evaluation

ACL 2025long

Repository-level code completion has drawn great attention in software engineering, and several benchmarks have been introduced. However, existing repository-level code completion benchmarks usually focus on a limited number of languages (<5), which cannot evaluate the general code intelligence abil…

2025

MPPO: Multi Pair-wise Preference Optimization for LLMs with Arbitrary Negative Samples

COLING 2025main

Aligning Large Language Models (LLMs) with human feedback is crucial for their development. Existing preference optimization methods such as DPO and KTO, while improved based on Reinforcement Learning from Human Feedback (RLHF), are inherently derived from PPO, requiring a reference model that adds…

Cited by 0SourcePDFScholar
2025

MTU-Bench: A Multi-granularity Tool-Use Benchmark for Large Language Models

ICLR 2025poster

Large Language Models (LLMs) have displayed massive improvements in reason- ing and decision-making skills and can hold natural conversations with users. Recently, many tool-use benchmark datasets have been proposed. However, existing datasets have the following limitations: (1). Insufficient evalua…

2025

Minimal Impact ControlNet: Advancing Multi-ControlNet Integration

ICLR 2025poster

With the advancement of diffusion models, there is a growing demand for high-quality, controllable image generation, particularly through methods that utilize one or multiple control signals based on ControlNet. However, in current ControlNet training, each control is designed to influence all areas…

Cited by 0SourcePDFScholar
2025

OmniKV: Dynamic Context Selection for Efficient Long-Context LLMs

ICLR 2025poster

During the inference phase of Large Language Models (LLMs) with long context, a substantial portion of GPU memory is allocated to the KV cache, with memory usage increasing as the sequence length grows. To mitigate the GPU memory footprint associate with KV cache, some previous studies have discarde…

2025

PGPO: Enhancing Agent Reasoning via Pseudocode-style Planning Guided Preference Optimization

ACL 2025finding

Large Language Model (LLM) agents have demonstrated impressive capabilities in handling complex interactive problems. Existing LLM agents mainly generate natural language plans to guide reasoning, which is verbose and inefficient. NL plans are also tailored to specific tasks and restrict agents’ abi…

2025

ProgCo: Program Helps Self-Correction of Large Language Models

ACL 2025short

Self-Correction aims to enable large language models (LLMs) to self-verify and self-refine their initial responses without external feedback. However, LLMs often fail to effectively self-verify and generate correct feedback, further misleading refinement and leading to the failure of self-correction…

2025

RHanDS: Refining Malformed Hands for Generated Images with Decoupled Structure and Style Guidance

AAAI 2025technical

Although diffusion models can generate high-quality human images, their applications are limited by the instability in generating hands with correct structures. In this paper, we introduce RHanDS, a conditional diffusion-based framework designed to refine malformed hands by utilizing decoupled struc…

2025

SMEC:Rethinking Matryoshka Representation Learning for Retrieval Embedding Compression

EMNLP 2025

Large language models (LLMs) generate high-dimensional embeddings that capture rich semantic and syntactic information. However, high-dimensional embeddings exacerbate computational complexity and storage requirements, thereby hindering practical deployment. To address these challenges, we propose a

Cited by 0SourcePDFScholar
2025

See the World, Discover Knowledge: A Chinese Factuality Evaluation for Large Vision Language Models

ACL 2025finding

The evaluation of factual accuracy in large vision language models (LVLMs) has lagged behind their rapid development, making it challenging to fully reflect these models’ knowledge capacity and reliability. In this paper, we introduce the first factuality-based visual question-answering benchmark in…

Cited by 0SourcePDFScholar
2025

Small Models are LLM Knowledge Triggers for Medical Tabular Prediction

ICLR 2025poster

Recent development in large language models (LLMs) has demonstrated impressive domain proficiency on unstructured textual or multi-modal tasks. However, despite with intrinsic world knowledge, their application on structured tabular data prediction still lags behind, primarily due to the numerical i…

2025

Think before Recommendation: Autonomous Reasoning-enhanced Recommender

NeurIPS 2025poster

The core task of recommender systems is to learn user preferences from historical user-item interactions. With the rapid development of large language models (LLMs), recent research has explored leveraging the reasoning capabilities of LLMs to enhance rating prediction tasks. However, existing disti…

Cited by 0SourceScholar
2025

Token Preference Optimization with Self-Calibrated Visual-Anchored Rewards for Hallucination Mitigation

EMNLP 2025

Direct Preference Optimization (DPO) has been demonstrated to be highly effective in mitigating hallucinations in Large Vision Language Models (LVLMs) by aligning their outputs more closely with human preferences. Despite the recent progress, existing methods suffer from two drawbacks: 1) Lack of sc

2025

Towards Visualization-of-Thought Jailbreak Attack against Large Visual Language Models

NeurIPS 2025poster

As Visual Language Models (VLMs) continue to evolve, they have demonstrated increasingly sophisticated logical reasoning capabilities and multimodal thought generation, opening doors to widespread applications. However, this advancement raises serious concerns about content security, particularly wh…

Cited by 0SourcecodeScholar
2025

TranSUN: A Preemptive Paradigm to Eradicate Retransformation Bias Intrinsically from Regression Models in Recommender Systems

NeurIPS 2025poster

Regression models are crucial in recommender systems. However, retransformation bias problem has been conspicuously neglected within the community. While many works in other fields have devised effective bias correction methods, all of them are post-hoc cures externally to the model, facing practica…

Cited by 0SourceScholar
2025

VC4VG: Optimizing Video Captions for Text-to-Video Generation

EMNLP 2025

Recent advances in text-to-video (T2V) generation highlight the critical role of high-quality video-text pairs in training models capable of producing coherent and instruction-aligned videos. However, strategies for optimizing video captions specifically for T2V training remain underexplored. In thi

2024

Accelerating Image Generation with Sub-path Linear Approximation Model

ECCV 2024oral

"Diffusion models have significantly advanced the state of the art in image, audio, and video generation tasks. However, their applications in practical scenarios are hindered by slow inference speed. Drawing inspiration from the consistency models, we propose the Sub-Path Linear Approximation Model…

Cited by 10SourcePDFScholar
2024

AuctionNet: A Novel Benchmark for Decision-Making in Large-Scale Games

NeurIPS 2024spotlight

Decision-making in large-scale games is an essential research area in artificial intelligence (AI) with significant real-world impact. However, the limited access to realistic large-scale game environments has hindered research progress in this area. In this paper, we present AuctionNet, a benchmark…

Cited by 3SourcecodeScholar
2024

ConceptMath: A Bilingual Concept-wise Benchmark for Measuring Mathematical Reasoning of Large Language Models

ACL 2024findings

This paper introduces ConceptMath, a bilingual (English and Chinese), fine-grained benchmark that evaluates concept-wise mathematical reasoning of Large Language Models (LLMs). Unlike traditional benchmarks that evaluate general mathematical reasoning with an average accuracy, ConceptMath systemical…

2024

D-CPT Law: Domain-specific Continual Pre-Training Scaling Law for Large Language Models

NeurIPS 2024poster

Continual Pre-Training (CPT) on Large Language Models (LLMs) has been widely used to expand the model’s fundamental understanding of specific downstream domains (e.g., math and code). For the CPT on domain-specific LLMs, one important question is how to choose the optimal mixture ratio between the g…

Cited by 18SourcePDFScholar
2024

DDK: Distilling Domain Knowledge for Efficient Large Language Models

NeurIPS 2024poster

Despite the advanced intelligence abilities of large language models (LLMs) in various applications, they still face significant computational and storage demands. Knowledge Distillation (KD) has emerged as an effective strategy to improve the performance of a smaller LLM (i.e., the student model)…

Cited by 10SourcePDFScholar
2024

Demystify Mamba in Vision: A Linear Attention Perspective

NeurIPS 2024poster

Mamba is an effective state space model with linear computation complexity. It has recently shown impressive efficiency in dealing with high-resolution inputs across various vision tasks. In this paper, we reveal that the powerful Mamba model shares surprising similarities with linear attention Tran…

2024

Dr.Hair: Reconstructing Scalp-Connected Hair Strands without Pre-Training via Differentiable Rendering of Line Segments

CVPR 2024highlight

In the film and gaming industries achieving a realistic hair appearance typically involves the use of strands originating from the scalp. However reconstructing these strands from observed surface images of hair presents significant challenges. The difficulty in acquiring Ground Truth (GT) data has…

Cited by 2SourcePDFScholar
2024

DynaInsRemover: A Real-time Dynamic Instance-Aware Static 3D LiDAR Mapping Framework for Dynamic Environment

ICRA 2024poster

Dynamic objects diversify the distribution of point cloud in the map, degrading the performance of the robotic downstream tasks. To address this problem, we present a novel real-time dynamic instance-aware static mapping framework called DynaInsRemover, which exploits the geometric discrepancies bet…

Cited by 2SourcecodeScholar
2024

E2-LLM: Efficient and Extreme Length Extension of Large Language Models

ACL 2024findings

Training Large Language Models (LLMs) to process extensive context lengths incurs prohibitive computational costs. Prevailing techniques for extending context capabilities in LLMs typically require not only additional training procedures but also access to datasets with long context (e.g., sequences…

Cited by 23SourcePDFScholar
2024

EMAGE: Towards Unified Holistic Co-Speech Gesture Generation via Expressive Masked Audio Gesture Modeling

CVPR 2024poster

We propose EMAGE a framework to generate full-body human gestures from audio and masked gestures encompassing facial local body hands and global movements. To achieve this we first introduce BEAT2 (BEAT-SMPLX-FLAME) a new mesh-level holistic co-speech dataset. BEAT2 combines a MoShed SMPL-X body wit…

2024

Exploring DCN-like architecture for fast image generation with arbitrary resolution

NeurIPS 2024poster

Arbitrary-resolution image generation still remains a challenging task in AIGC, as it requires handling varying resolutions and aspect ratios while maintaining high visual quality. Existing transformer-based diffusion methods suffer from quadratic computation cost and limited resolution extrapolatio…

Cited by 0SourcePDFScholar
2024

GeoGPT4V: Towards Geometric Multi-modal Large Language Models with Geometric Image Generation

EMNLP 2024main

Large language models have seen widespread adoption in math problem-solving, yet for geometry problems, which often necessitate visual aids even for humans, the most advanced multi-modal models still struggle to effectively utilize image information. High-quality data is crucial for enhancing the ge…

2024

GraphReader: Building Graph-based Agent to Enhance Long-Context Abilities of Large Language Models

EMNLP 2024finding

Long-context capabilities are essential for large language models (LLMs) to tackle complex and long-input tasks. Despite numerous efforts made to optimize LLMs for long contexts, challenges persist in robustly processing long inputs. In this paper, we introduce GraphReader, a graph-based agent syste…

2024

MT-Bench-101: A Fine-Grained Benchmark for Evaluating Large Language Models in Multi-Turn Dialogues

ACL 2024long

The advent of Large Language Models (LLMs) has drastically enhanced dialogue systems. However, comprehensively evaluating the dialogue abilities of LLMs remains a challenge. Previous benchmarks have primarily focused on single-turn dialogues or provided coarse-grained and incomplete assessments of m…

2024

Making Pre-trained Language Models Great on Tabular Prediction

ICLR 2024spotlight

The transferability of deep neural networks (DNNs) has made significant progress in image and language processing. However, due to the heterogeneity among tables, such DNN bonus is still far from being well exploited on tabular data prediction (e.g., regression or classification tasks). Condensing k…

2024

Percentile Risk-Constrained Budget Pacing for Guaranteed Display Advertising in Online Optimization

AAAI 2024technical

Guaranteed display (GD) advertising is a critical component of advertising since it provides publishers with stable revenue and enables advertisers to target specific audiences with guaranteed impressions. However, smooth pacing control for online ad delivery presents a challenge due to significant…

2024

SEGMENT+: Long Text Processing with Short-Context Language Models

EMNLP 2024main

There is a growing interest in expanding the input capacity of language models (LMs) across various domains. However, simply increasing the context window does not guarantee robust performance across diverse long-input processing tasks, such as understanding extensive documents and extracting detail…

Cited by 1SourcePDFScholar
2024

Synchronized Video Storytelling: Generating Video Narrations with Structured Storyline

ACL 2024long

Video storytelling is engaging multimedia content that utilizes video and its accompanying narration to share a story and attract the audience, where a key challenge is creating narrations for recorded visual scenes. Previous studies on dense video captioning and video story generation have made som…

2024

Turbo: Informativity-Driven Acceleration Plug-In for Vision-Language Large Models

ECCV 2024oral

"Vision-Language Large Models (VLMs) recently become primary backbone of AI, due to the impressive performance. However, their expensive computation costs, i.e., throughput and delay, impede potentials in the real-world scenarios. To achieve acceleration for VLMs, most existing methods focus on the…

Cited by 9SourcePDFScholar
2023

Co-optimization of Morphology and Behavior of Modular Robots via Hierarchical Deep Reinforcement Learning

RSS 2023poster

Modular robots hold the promise of changing their shape and even dimension to adapt to various tasks and environments. To realize this superiority, it is essential to find the appropriate morphology and its corresponding behavior simultaneously to ensure optimality of the reconfiguration. However, a…

Cited by 11SourcePDFScholar
2023

Combating Bilateral Edge Noise for Robust Link Prediction

NeurIPS 2023poster

Although link prediction on graphs has achieved great success with the development of graph neural networks (GNNs), the potential robustness under the edge noise is still less investigated. To close this gap, we first conduct an empirical study to disclose that the edge noise bilaterally perturbs bo…

2023

Exploiting Prompt Learning with Pre-Trained Language Models for Alzheimer's Disease Detection

ICASSP 2023accepted

Early diagnosis of Alzheimer’s disease (AD) is crucial in facilitating preventive care and to delay further progression. Speech based automatic AD screening systems provide a non-intrusive and more scalable alternative to other clinical screening techniques. Textual embedding features produced by pr…

Cited by 0SourceScholar
2023

Leveraging Pretrained Representations With Task-Related Keywords for Alzheimer's Disease Detection

ICASSP 2023accepted

With the global population aging rapidly, Alzheimer’s disease (AD) is particularly prominent in older adults, which has an insidious onset and leads to a gradual, irreversible deterioration in cognitive domains (memory, communication, etc.). Speech-based AD detection opens up the possibility of wide…

Cited by 0SourceScholar
2022

APG: Adaptive Parameter Generation Network for Click-Through Rate Prediction

NeurIPS 2022accept

In many web applications, deep learning-based CTR prediction models (deep CTR models for short) are widely adopted. Traditional deep CTR models learn patterns in a static manner, i.e., the network parameters are the same across all the instances. However, such a manner can hardly characterize each…

Cited by 37SourcePDFScholar
2022

BEAT: A Large-Scale Semantic and Emotional Multi-modal Dataset for Conversational Gestures Synthesis

ECCV 2022poster

"Achieving realistic, vivid, and human-like synthesized conversational gestures conditioned on multi-modal data is still an unsolved problem due to the lack of available datasets, models and standard evaluation metrics. To address this, we build Body-Expression-Audio-Text dataset, BEAT, which has i)…

2022

CREATER: CTR-driven Advertising Text Generation with Controlled Pre-Training and Contrastive Fine-Tuning

NAACL 2022industry

This paper focuses on automatically generating the text of an ad, and the goal is that the generated text can capture user interest for achieving higher click-through rate (CTR). We propose CREATER, a CTR-driven advertising text generation approach, to generate ad texts based on high-quality user re…

2022

Characterizing the Adversarial Vulnerability of Speech self-Supervised Learning

ICASSP 2022accepted

A leaderboard named Speech processing Universal PERformance Benchmark (SUPERB), which aims at benchmarking the performance of a shared self-supervised learning (SSL) speech model across various downstream speech tasks with minimal modification of architectures and a small amount of data, has fueled…

Cited by 0SourceScholar
2022

GBA: A Tuning-free Approach to Switch between Synchronous and Asynchronous Training for Recommendation Models

NeurIPS 2022accept

High-concurrency asynchronous training upon parameter server (PS) architecture and high-performance synchronous training upon all-reduce (AR) architecture are the most commonly deployed distributed training modes for recommendation models. Although synchronous AR training is designed to have higher…

Cited by 3SourcePDFScholar
2022

StableMoE: Stable Routing Strategy for Mixture of Experts

ACL 2022long

The Mixture-of-Experts (MoE) technique can scale up the model size of Transformers with an affordable computational overhead. We point out that existing learning-to-route MoE methods suffer from the routing fluctuation issue, i.e., the target expert of the same input may change along with training,…

2022

Sustainable Online Reinforcement Learning for Auto-bidding

NeurIPS 2022accept

Recently, auto-bidding technique has become an essential tool to increase the revenue of advertisers. Facing the complex and ever-changing bidding environments in the real-world advertising system (RAS), state-of-the-art auto-bidding policies usually leverage reinforcement learning (RL) algorithms t…

2022

XLM-E: Cross-lingual Language Model Pre-training via ELECTRA

ACL 2022long

In this paper, we introduce ELECTRA-style tasks to cross-lingual language model pre-training. Specifically, we present two pre-training tasks, namely multilingual replaced token detection, and translation replaced token detection. Besides, we pretrain the model, named as XLM-E, on both multilingual…

2021

4DComplete: Non-Rigid Motion Estimation Beyond the Observable Surface

ICCV 2021poster

Tracking non-rigidly deforming scenes using range sensors has numerous applications including computer vision, AR/VR, and robotics. However, due to occlusions and physical limitations of range sensors, existing methods only handle the visible surface, thus causing discontinuities and incompleteness…

Cited by 117PDFcodeScholar
2021

Allocating Large Vocabulary Capacity for Cross-Lingual Language Model Pre-Training

EMNLP 2021main

Compared to monolingual models, cross-lingual models usually require a more expressive vocabulary to represent all languages adequately. We find that many languages are under-represented in recent cross-lingual language models due to the limited vocabulary capacity. To this end, we propose an algori…

2021

Consistency Regularization for Cross-Lingual Fine-Tuning

ACL 2021long

Fine-tuning pre-trained cross-lingual language models can transfer task-specific supervision from one language to the others. In this work, we propose to improve cross-lingual fine-tuning with consistency regularization. Specifically, we use example consistency regularization to penalize the predict…

2021

Improving Pretrained Cross-Lingual Language Models via Self-Labeled Word Alignment

ACL 2021long

The cross-lingual language models are typically pretrained with masked language modeling on multilingual text or parallel sentences. In this paper, we introduce denoising word alignment as a new cross-lingual pre-training task. Specifically, the model first self-label word alignments for parallel se…

2020

Efficient Spatio-Temporal Recurrent Neural Network for Video Deblurring

ECCV 2020poster

Real-time video deblurring still remains a challenging task due to the complexity of spatially and temporally varying blur itself and the requirement of low computational cost. To improve the network efficiency, we adopt residual dense blocks into RNN cells, so as to efficiently extract the spatial…

Cited by 175SourcePDFScholar
2020

Stereoscopic Flash and No-Flash Photography for Shape and Albedo Recovery

CVPR 2020poster

We present a minimal imaging setup that harnesses both geometric and photometric approaches for shape and albedo recovery. We adopt a stereo camera and a flashlight to capture a stereo image pair and a flash/no-flash pair. From the stereo image pair, we recover a rough shape that captures low-freque…

Cited by 12PDFScholar