← Search

Zihan Wang

111 accepted papers

2026

AetherCode: Evaluating LLMs’ Ability to Win In Premier Programming Competitions

ICLR 2026poster

Competitive programming has emerged as a critical benchmark for evaluating the reasoning and coding capabilities of Large Language Models (LLMs). Despite impressive progress on existing benchmarks, we argue that current evaluations overstate model proficiency, masking a substantial gap between LLMs…

Cited by 0SourceScholar
2026

Benchmarking Agent Memory in Interdependent Multi-Session Agentic Tasks

ICML 2026poster

Existing evaluations of agents with memory typically assess **memorization** and **action** in isolation. One class of benchmarks evaluates memorization by testing recall of past conversations or text but fails to capture how memory is used to guide future decisions. Another class focuses on agents …

Cited by 0SourceScholar
2026

ChartNet: A Million-Scale, High-Quality Multimodal Dataset for Robust Chart Understanding

CVPR 2026

Understanding charts requires models to jointly reason over geometric visual patterns, structured numerical data, and natural language -- a capability where current vision-language models (VLMs) remain limited. We introduce ChartNet, a high-quality, million-scale multimodal dataset designed to advan

Cited by 0SourceScholar
2026

CloserToMe: A Unified Framework for Accurate and Transferable Latency Prediction Across Heterogeneous Devices

AAAI 2026technical

Hardware accelerators such as GPUs, NPUs, and FPGAs are essential to meeting AI’s computational demands. With the proliferation of heterogeneous devices across cloud and edge, various model optimization techniques adapt to diverse hardware characteristics through operator transformations and structu

Cited by 0SourcePDFScholar
2026

CompBench: Benchmarking Complex Instruction-guided Image Editing

CVPR 2026

While real-world applications increasingly demand intricate scene manipulation, existing instruction-guided image editing benchmarks often oversimplify task complexity and lack comprehensive, fine-grained instructions. To bridge this gap, we introduce CompBench, a large-scale benchmark specifically

Cited by 0SourcecodeScholar
2026

Constitutional Classifiers++: Production-Grade Defenses against Universal Jailbreaks

ICLR 2026poster

We introduce enhanced Constitutional Classifiers that deliver production-grade jailbreak robustness with dramatically reduced computational costs and refusal rates compared to previous-generation defenses. We first identify vulnerabilities in existing systems that evaluate model outputs without rega…

Cited by 0SourceScholar
2026

Contact-guided Real2Sim from Monocular Video with Planar Scene Primitives

ICLR 2026poster

We introduce CRISP, a method that recovers simulatable human motion and scene geometry from monocular video. Prior work on joint human--scene reconstruction relies on data-driven priors and joint optimization with no physics in the loop, or recovers noisy geometry with artifacts that cause motion-tr…

Cited by 0SourcecodeScholar
2026

D3D-VLP: Dynamic 3D Vision-Language-Planning Model for Embodied Grounding and Navigation

CVPR 2026

Embodied agents face a critical dilemma that end-to-end models lack interpretability and explicit 3D reasoning, while modular systems ignore cross-component interdependencies and synergies. To bridge this gap, we propose the Dynamic 3D Vision-Language-Planning Model (D3D-VLP). Our model introduces t

Cited by 0SourcecodeScholar
2026

E-mem: Multi-Agent Based Episodic Context Reconstruction for LLM Agent Memory

ICML 2026poster

The evolution of Large Language Model (LLM) agents towards System~2 reasoning, characterized by deliberative, high-precision problem-solving, necessitates maintaining rigorous logical integrity over extended horizons. However, prevalent memory preprocessing paradigms incur destructive de-contextuali…

Cited by 0SourceScholar
2026

ECG-R1: Protocol-Guided and Modality-Agnostic MLLM for Reliable ECG Interpretation

ICML 2026poster

Electrocardiography (ECG) serves as an indispensable diagnostic tool in clinical practice, yet existing multimodal large language models (MLLMs) remain unreliable for ECG interpretation, often producing plausible but clinically incorrect analyses. To address this, we propose ECG-R1, the first reason…

Cited by 0SourceScholar
2026

Efficient Multimodal Spatial Reasoning via Dynamic and Asymmetric Routing

ICLR 2026poster

Recently, visualization-of-thought (VoT) has unlocked new opportunities for complex spatial reasoning in multimodal large language models (MLLMs) by complementing verbal reasoning with visual thinking. However, the autoregressive accumulation of lengthy and redundant tokens substantially increases c…

Cited by 0SourceScholar
2026

EmbodiedSplat: Online Feed-Forward Semantic 3DGS for Open-Vocabulary 3D Scene Understanding

CVPR 2026

Understanding a 3D scene immediately with its exploration is essential for embodied tasks, where an agent must construct and comprehend the 3D representation in an online and nearly real-time manner. In this study, we propose EmbodiedSplat, an online feed-forward 3DGS for open-vocabulary scene under

Cited by 0SourcecodeScholar
2026

From Parameter Dynamics to Risk Scoring: Quantifying Sample-Level Safety Degradation in LLM Fine-tuning

ICML 2026poster

Safety alignment of Large Language Models (LLMs) is extremely fragile, fine-tuning on small number of benign samples can erase safety behaviors learned from millions of preference examples. Existing studies attempt to explain this phenomenon by comparing parameters and hidden states before and after…

Cited by 0SourceScholar
2026

GA-VLN: Geometry-Aware BEV Representation for Efficient Vision-Language Navigation

CVPR 2026

Despite significant progress in Vision-Language Navigation (VLN), existing approaches still rely on dense RGB videos that produce excessive patch tokens and lack explicit spatial structure, resulting in substantial computational overhead and limited spatial reasoning. To address these issues, we int

Cited by 0SourceScholar
2026

GTPO and GRPO-S: Token and Sequence-Level Reward Shaping with Policy Entropy

ICML 2026poster

Reinforcement Learning (RL) is pivotal for enhancing Large Language Model (LLM) reasoning, yet mainstream algorithms such as GRPO and DAPO remain constrained by a coarse-grained credit assignment paradigm, where all tokens within the same response receive the identical reward. In this paper, we prop…

Cited by 0SourceScholar
2026

History to Future: Evolving Agent with Experience and Thought for Zero-shot Vision-and-Language Navigation

CVPR 2026

Vision-and-Language Navigation in Continuous Environment (VLN-CE) requires an agent to follow language instructions to navigate the target destination. With the advancement of large language models (LLMs), recent efforts have explored adapting them for zero-shot VLN-CE, offering a promising solution

Cited by 0SourceScholar
2026

KeepKV: Achieving Periodic Lossless KV Cache Compression for Efficient LLM Inference

AAAI 2026technical

Efficient inference of large language models (LLMs) is hindered by an ever-growing key-value (KV) cache, making KV cache compression a critical research direction. Traditional methods selectively evict less important KV cache entries, which leads to information loss and hallucinations. Recently, mer

Cited by 0SourcePDFScholar
2026

MPMA: Preference Manipulation Attack Against Model Context Protocol

AAAI 2026technical

Model Context Protocol (MCP) standardizes interface mapping for large language models (LLMs) to access external data and tools, which revolutionizes the paradigm of tool selection and facilitates the rapid expansion of the LLM agent tool ecosystem. However, as the MCP is increasingly adopted, third-

Cited by 0SourcePDFScholar
2026

MTAttack: Multi-Target Backdoor Attacks Against Large Vision-Language Models

AAAI 2026technical

Recent advances in Large Visual Language Models (LVLMs) have demonstrated impressive performance across various vision-language tasks by leveraging large-scale image-text pretraining and instruction tuning. However, the security vulnerabilities of LVLMs have become increasingly concerning, particula

Cited by 0SourcePDFScholar
2026

NL2Repo-Bench: Towards Long-Horizon Repository Generation Evaluation of Coding Agents

ICML 2026poster

Recent advances in coding agents suggest rapid progress toward autonomous software development, yet existing benchmarks primarily evaluate short-horizon behaviors such as localized code generation, scaffolded completion, or repository repair, leaving it unclear whether agents can sustain coherent re…

Cited by 0SourceScholar
2026

STELAR-VISION: Self-Topology-Aware Efficient Learning for Aligned Reasoning in Vision

AAAI 2026technical

Vision-language models (VLMs) have made significant strides in reasoning, yet they often struggle with complex multimodal tasks and tend to generate overly verbose outputs. A key limitation is their reliance on chain-of-thought (CoT) reasoning, despite many tasks benefiting from alternative topologi

Cited by 0SourcePDFScholar
2026

Theory of Space: Can Foundation Models Construct Spatial Beliefs through Active Exploration?

ICLR 2026poster

Spatial embodied intelligence often operates under partial observability, where agents must act to acquire missing information rather than passively consume complete observations. In such settings, progress depends on actively selecting informative actions that reduce uncertainty and support the con…

Cited by 0SourcecodeScholar
2026

TiViBench: Benchmarking Think-in-Video Reasoning for Video Generation

CVPR 2026

The rapid evolution of video generative models has shifted their focus from producing visually plausible outputs to tackling tasks requiring physical plausibility and logical consistency. However, despite recent breakthroughs such as Veo 3's chain-of-frames reasoning, it remains unclear whether thes

Cited by 0SourcecodeScholar
2026

Understanding VLMs Spatial Mental Modeling Capability from Limited Views

ICLR 2026poster

Can Vision Language Models (VLMs) imagine the full scene from just a few views, like humans do? Humans form spatial mental models, internal representations of unseen space, to reason about layout, perspective, and motion. Our new MindCube benchmark with 21,154 questions across 3,268 images exposes t…

Cited by 0SourcecodeScholar
2026

WET: Mitigating World-Conditioned Knowledge Conflicts via World Entropy Tethering

ICML 2026poster

Large language models (LLMs) face a "loyalty dilemma" when correctness is conditioned on an active world-of-discourse. We identify a systemic failure mode---world misattribution---where models implicitly ground generation in an incompatible regime and drift from the target world. We propose World En…

Cited by 0SourceScholar
2026

WFR-FM: Simulation-Free Dynamic Unbalanced Optimal Transport

ICLR 2026poster

The Wasserstein–Fisher–Rao (WFR) metric extends dynamic optimal transport (OT) by coupling displacement with change of mass, providing a principled geometry for modeling unbalanced snapshot dynamics. Existing WFR solvers, however, are often unstable, computationally expensive, and difficult to scale…

Cited by 0SourcecodeScholar
2026

Weak-to-Strong Generalization with Failure Trajectories

ICLR 2026poster

Weak-to-Strong generalization (W2SG) is a new trend to elicit the full capabilities of a strong model with supervision from a weak model. While existing W2SG studies focus on simple tasks like binary classification, we extend this paradigm to complex interactive decision-making environments. Speci…

Cited by 0SourcecodeScholar
2026

Your Language Model Secretly Contains Personality Subnetworks

ICLR 2026poster

Humans shift between different personas depending on social context. Large Language Models (LLMs) demonstrate a similar flexibility in adopting different personas and behaviors. Existing approaches, however, typically adapt such behavior through external knowledge such as prompting, retrieval-augmen…

Cited by 0SourcecodeScholar
2025

Alignment for Efficient Tool Calling of Large Language Models

EMNLP 2025

Recent advancements in tool learning have enabled large language models (LLMs) to integrate external tools, enhancing their task performance by expanding their knowledge boundaries. However, relying on tools often introduces trade-offs between performance, speed, and cost, with LLMs sometimes exhibi

Cited by 0SourcePDFScholar
2025

Auxiliary Prompt Tuning of Vision-Language Models for Few-Shot Out-of-Distribution Detection

ICCV 2025poster

Recent advancements in CLIP-based out-of-distribution (OOD) detection have shown promising results via regularization on prompt tuning, leveraging background features extracted from a few in-distribution (ID) samples as proxies for OOD features.However, these methods suffer from an inherent limitati…

2025

CodeContests+: High-Quality Test Case Generation for Competitive Programming

EMNLP 2025

Competitive programming, due to its high reasoning difficulty and precise correctness feedback, has become a key task for both training and evaluating the reasoning capabilities of large language models (LLMs). However, while a large amount of public problem data, such as problem statements and solu

Cited by 0SourcePDFScholar
2025

Dynam3D: Dynamic Layered 3D Tokens Empower VLM for Vision-and-Language Navigation

NeurIPS 2025oral

Vision-and-Language Navigation (VLN) is a core task where embodied agents leverage their spatial mobility to navigate in 3D environments toward designated destinations based on natural language instructions. Recently, video-language large models (Video-VLMs) with strong generalization capabilities a…

Cited by 0SourcecodeScholar
2025

Elastic Representation: Mitigating Spurious Correlations for Group Robustness

AISTATS 2025poster

Deep learning models can suffer from severe performance degradation when relying on spurious correlations between input features and labels, making the models perform well on training data but have poor prediction accuracy for minority groups. This problem arises especially when training data are li…

Cited by 0SourceScholar
2025

InfoGain-RAG: Boosting Retrieval-Augmented Generation through Document Information Gain-based Reranking and Filtering

EMNLP 2025

Retrieval-Augmented Generation (RAG) has emerged as a promising approach to address key limitations of Large Language Models (LLMs), such as hallucination, outdated knowledge, and lacking reliable reference. However, current RAG frameworks often struggle with identifying whether retrieved documents

Cited by 0SourcePDFScholar
2025

InstructPart: Task-Oriented Part Segmentation with Instruction Reasoning

ACL 2025long

Large multimodal foundation models, particularly in the domains of language and vision, have significantly advanced various tasks, including robotics, autonomous driving, information retrieval, and grounding. However, many of these models perceive objects as indivisible, overlooking the components t…

2025

Knowledge Transfer Across Modalities for Weakly Supervised Point Cloud Semantic Segmentation

ICASSP 2025accepted

Current weakly supervised point cloud semantic segmentation struggles with insufficient utilization of limited annotations in unimodal representation learning due to the sparse and textureless nature of point clouds. In this work, we leverage cross-modality information by transferring knowledge from…

Cited by 0SourceScholar
2025

MACPO: Weak-to-Strong Alignment via Multi-Agent Contrastive Preference Optimization

ICLR 2025poster

As large language models (LLMs) are rapidly advancing and achieving near-human capabilities on specific tasks, aligning them with human values is becoming more urgent. In scenarios where LLMs outperform humans, we face a weak-to-strong alignment problem where we need to effectively align strong stud…

Cited by 3SourcePDFScholar
2025

MUSE: A Multimodal Conversational Recommendation Dataset with Scenario-Grounded User Profiles

ACL 2025finding

Current conversational recommendation systems focus predominantly on text. However, real-world recommendation settings are generally multimodal, causing a significant gap between existing research and practical applications. To address this issue, we propose Muse, the first multimodal conversational…

Cited by 0SourcePDFScholar
2025

Modeling Cell Dynamics and Interactions with Unbalanced Mean Field Schrödinger Bridge

NeurIPS 2025poster

Modeling the dynamics from sparsely time-resolved snapshot data is crucial for understanding complex cellular processes and behavior. Existing methods leverage optimal transport, Schrödinger bridge theory, or their variants to simultaneously infer stochastic, unbalanced dynamics from snapshot data.…

Cited by 0SourcecodeScholar
2025

MonoFusion: Sparse-View 4D Reconstruction via Monocular Fusion

ICCV 2025poster

We address the problem of dynamic scene reconstruction from sparse-view videos. Prior work often requires dense multi-view captures with hundreds of calibrated cameras (e.g. Panoptic Studio) - such multi-view setups are prohibitively expensive to build and cannot capture diverse scenes in-the-wild.…

Cited by 0SourcePDFScholar
2025

NavRAG: Generating User Demand Instructions for Embodied Navigation through Retrieval-Augmented LLM

ACL 2025finding

Vision-and-Language Navigation (VLN) is an essential skill for embodied agents, allowing them to navigate in 3D environments following natural language instructions. High-performance navigation models require a large amount of training data, the high cost of manually annotating data has seriously hi…

2025

Re-thinking Temporal Search for Long-Form Video Understanding

CVPR 2025poster

Efficient understanding of long-form videos remains a significant challenge in computer vision. In this work, we revisit temporal search paradigms for long-form video understanding, studying a fundamental issue pertaining to all state-of-the-art (SOTA) long-context vision-language models (VLMs). In…

2025

Reducing Tool Hallucination via Reliability Alignment

ICML 2025poster

Large Language Models (LLMs) have expanded their capabilities beyond language generation to interact with external tools, enabling automation and real-world applications. However, tool hallucinations—where models either select inappropriate tools or misuse them—pose significant challenges, leading t…

Cited by 6SourcePDFScholar
2025

VAGEN: Reinforcing World Model Reasoning for Multi-Turn VLM Agents

NeurIPS 2025poster

A major challenge in training VLM agents, compared to LLM agents, is that states shift from simple texts to complex visual observations, which introduces partial observability and demands robust world modeling. We ask: can VLM agents build internal world models through explicit visual state reasonin…

Cited by 0SourceScholar
2025

Variational Regularized Unbalanced Optimal Transport: Single Network, Least Action

NeurIPS 2025poster

Recovering the dynamics from a few snapshots of a high-dimensional system is a challenging task in statistical physics and machine learning, with important applications in computational biology. Many algorithms have been developed to tackle this problem, based on frameworks such as optimal transport…

Cited by 0SourcecodeScholar
2025

Video-MME: The First-Ever Comprehensive Evaluation Benchmark of Multi-modal LLMs in Video Analysis

CVPR 2025highlight

In the quest for artificial general intelligence, Multi-modal Large Language Models (MLLMs) have emerged as a focal point in recent advancements. However, the predominant focus remains on developing their capabilities in static image understanding. The potential of MLLMs to process sequential visual…

Cited by 368SourcePDFScholar
2025

Wide Neural Networks Trained with Weight Decay Provably Exhibit Neural Collapse

ICLR 2025oral

Deep neural networks (DNNs) at convergence consistently represent the training data in the last layer via a geometric structure referred to as neural collapse. This empirical evidence has spurred a line of theoretical research aimed at proving the emergence of neural collapse, mostly focusing on the…

Cited by 2SourcePDFScholar
2024

AirShot: Efficient Few-Shot Detection for Autonomous Exploration

IROS 2024poster

Few-shot object detection has drawn increasing attention in the field of robotic exploration, where robots are required to find unseen objects with a few online provided examples. Despite recent efforts have been made to yield online processing capabilities, slow inference speeds of low-powered robo…

Cited by 8SourcecodeScholar
2024

Answer is All You Need: Instruction-following Text Embedding via Answering the Question

ACL 2024long

This work aims to build a text embedder that can capture characteristics of texts specified by user instructions clarifying the similarity criterion. While previous methods improve general task awareness by injecting the instruction information into encoding, they fail to be sensitive to clearer cri…

2024

ChatGLM-Math: Improving Math Problem-Solving in Large Language Models with a Self-Critique Pipeline

EMNLP 2024finding

Large language models (LLMs) have shown excellent mastering of human language but still struggle in real-world applications that require mathematical problem-solving. While many strategies and datasets to enhance LLMs’ mathematics are developed, it remains a challenge to simultaneously maintain and…

2024

CogAgent: A Visual Language Model for GUI Agents

CVPR 2024highlight

People are spending an enormous amount of time on digital devices through graphical user interfaces (GUIs) e.g. computer or smartphone screens. Large language models (LLMs) such as ChatGPT can assist people in tasks like writing emails but struggle to understand and interact with GUIs thus limiting…

2024

Evaluating the Smooth Control of Attribute Intensity in Text Generation with LLMs

ACL 2024findings

Controlling the attribute intensity of text generation is crucial across scenarios (e.g., writing conciseness, chatting emotion, and explanation clarity). The remarkable capabilities of large language models (LLMs) have revolutionized text generation, prompting us to explore such smooth control of L…

2024

Evoke: Evoking Critical Thinking Abilities in LLMs via Reviewer-Author Prompt Editing

ICLR 2024poster

Large language models (LLMs) have made impressive progress in natural language processing. These models rely on proper human instructions (or prompts) to generate suitable responses. However, the potential of LLMs are not fully harnessed by commonly-used prompting methods: many human-in-the-loop alg…

Cited by 7SourcePDFScholar
2024

I Can Tell What I am Doing: Toward Real-World Natural Language Grounding of Robot Experiences

CoRL 2024poster

Understanding robot behaviors and experiences through natural language is crucial for developing intelligent and transparent robotic systems. Recent advancement in large language models (LLMs) makes it possible to translate complex, multi-modal robotic experiences into coherent, human-readable narra…

Cited by 4SourceScholar
2024

Implicit bias of SGD in $L_2$-regularized linear DNNs: One-way jumps from high to low rank

ICLR 2024spotlight

The $L_{2}$-regularized loss of Deep Linear Networks (DLNs) with more than one hidden layers has multiple local minima, corresponding to matrices with different ranks. In tasks such as matrix completion, the goal is to converge to the local minimum with the smallest rank that still fits the training…

Cited by 20SourcePDFScholar
2024

Learn from Failure: Fine-tuning LLMs with Trial-and-Error Data for Intuitionistic Propositional Logic Proving

ACL 2024long

Recent advances in Automated Theorem Proving have shown the effectiveness of leveraging a (large) language model that generates tactics (i.e. proof steps) to search through proof states. The current model, while trained solely on successful proof paths, faces a discrepancy at the inference stage, as…

2024

Learning Fair Representations for Recommendation via Information Bottleneck Principle

IJCAI 2024poster

User-oriented recommender systems (RS) characterize users' preferences based on observed behaviors and are widely deployed in personalized services. However, RS may unintentionally capture biases related to sensitive attributes (e.g., gender) from behavioral data, leading to unfair issues and discri…

2024

Let the Expert Stick to His Last: Expert-Specialized Fine-Tuning for Sparse Architectural Large Language Models

EMNLP 2024main

Parameter-efficient fine-tuning (PEFT) is crucial for customizing Large Language Models (LLMs) with constrained resource. Although there have been various PEFT methods for dense-architecture LLMs, PEFT for sparse-architecture LLMs is still underexplored. In this work, we study the PEFT method for LL…

2024

Lookahead Exploration with Neural Radiance Representation for Continuous Vision-Language Navigation

CVPR 2024highlight

Vision-and-language navigation (VLN) enables the agent to navigate to a remote location following the natural language instruction in 3D environments. At each navigation step the agent selects from possible candidate locations and then makes the move. For better navigation planning the lookahead exp…

2024

MINT: Evaluating LLMs in Multi-turn Interaction with Tools and Language Feedback

ICLR 2024poster

To solve complex tasks, large language models (LLMs) often require multiple rounds of interactions with the user, sometimes assisted by external tools. However, current evaluation protocols often emphasize benchmark performance with single-turn exchanges, neglecting the nuanced interactions among th…

Cited by 135SourcePDFScholar
2024

Multi-step Problem Solving Through a Verifier: An Empirical Analysis on Model-induced Process Supervision

EMNLP 2024finding

Process supervision, using a trained verifier to evaluate the intermediate steps generated by a reasoner, has demonstrated significant improvements in multi-step problem solving. In this paper, to avoid the expensive effort of human annotation on the verifier training data, we introduce Model-induce…

Cited by 38SourcePDFScholar
2024

SciCode: A Research Coding Benchmark Curated by Scientists

NeurIPS 2024poster

Since language models (LMs) now outperform average humans on many challenging tasks, it is becoming increasingly difficult to develop challenging, high-quality, and realistic evaluations. We address this by examining LM capabilities to generate code for solving real scientific research problems. Inc…

Cited by 18SourcePDFScholar
2024

SciInstruct: a Self-Reflective Instruction Annotated Dataset for Training Scientific Language Models

NeurIPS 2024poster

Large Language Models (LLMs) have shown promise in assisting scientific discovery. However, such applications are currently limited by LLMs' deficiencies in understanding intricate scientific concepts, deriving symbolic equations, and solving advanced numerical calculations. To bridge these gaps, we…

2024

Self-Renewal Prompt Optimizing with Implicit Reasoning

EMNLP 2024finding

The effectiveness of Large Language Models (LLMs) relies on their capacity to understand instructions and generate human-like responses. However, aligning LLMs with complex human preferences remains a significant challenge due to the potential misinterpretation of user prompts. Current methods for a…

Cited by 0SourcePDFScholar
2024

Sim-to-Real Transfer via 3D Feature Fields for Vision-and-Language Navigation

CoRL 2024poster

Vision-and-language navigation (VLN) enables the agent to navigate to a remote location in 3D environments following the natural language instruction. In this field, the agent is usually trained and evaluated in the navigation simulators, lacking effective approaches for sim-to-real transfer. The VL…

Cited by 11SourcecodeScholar
2024

Synergistic Reinforcement and Imitation Learning for Vision-driven Autonomous Flight of UAV Along River

IROS 2024poster

Vision-driven autonomous flight and obstacle avoidance of Unmanned Aerial Vehicles (UAVs) along complex riverine environments for tasks like rescue and surveillance requires a robust navigation policy, which is yet difficult to obtain due to the shortage of trainable riverine environment simulators.…

Cited by 1SourcecodeScholar
2024

Text Grafting: Near-Distribution Weak Supervision for Minority Classes in Text Classification

EMNLP 2024main

For extremely weak-supervised text classification, pioneer research generates pseudo labels by mining texts similar to the class names from the raw corpus, which may end up with very limited or even no samples for the minority classes. Recent works have started to generate the relevant texts by prom…

2024

Utilizing Local Hierarchy with Adversarial Training for Hierarchical Text Classification

COLING 2024main

Hierarchical text classification (HTC) is a challenging subtask of multi-label classification due to its complex taxonomic structure. Nearly all recent HTC works focus on how the labels are structured but ignore the sub-structure of ground-truth labels according to each input text which contains fru…

2023

A Benchmark on Extremely Weakly Supervised Text Classification: Reconcile Seed Matching and Prompting Approaches

ACL 2023findings

Extremely Weakly Supervised Text Classification (XWS-TC) refers to text classification based on minimal high-level human guidance, such as a few label-indicative seed words or classification instructions. There are two mainstream approaches for XWS-TC, however, never being rigorously compared: (1) t…

2023

Autonomous Robotic Reinforcement Learning with Asynchronous Human Feedback

CoRL 2023poster

Ideally, we would place a robot in a real-world environment and leave it there improving on its own by gathering more experience autonomously. However, algorithms for autonomous robotic learning have been challenging to realize in the real world. While this has often been attributed to the challenge…

Cited by 6SourcecodeScholar
2023

Breadcrumbs to the Goal: Goal-Conditioned Exploration from Human-in-the-Loop Feedback

NeurIPS 2023poster

Exploration and reward specification are fundamental and intertwined challenges for reinforcement learning. Solving sequential decision making tasks with a non-trivial element of exploration requires either specifying carefully designed reward functions or relying on indiscriminate, novelty seeking…

2023

Debiasing Made State-of-the-art: Revisiting the Simple Seed-based Weak Supervision for Text Classification

EMNLP 2023long main

Recent advances in weakly supervised text classification mostly focus on designing sophisticated methods to turn high-level human heuristics into quality pseudo-labels. In this paper, we revisit the seed matching-based method, which is arguably the simplest way to generate pseudo-labels, and show th…

Cited by 0SourcecodeScholar
2023

GLM-130B: An Open Bilingual Pre-trained Model

ICLR 2023poster

We introduce GLM-130B, a bilingual (English and Chinese) pre-trained language model with 130 billion parameters. It is an attempt to open-source a 100B-scale model as good as GPT-3 (davinci) and unveil how models of such a scale can be successfully pre-trained. Over the course of this effort, we fac…

2023

Generalizing Few-Shot Named Entity Recognizers to Unseen Domains with Type-Related Features

EMNLP 2023long findings

Few-shot named entity recognition (NER) has shown remarkable progress in identifying entities in low-resource domains. However, few-shot NER methods still struggle with out-of-domain (OOD) examples due to their reliance on manual labeling for the target domain. To address this limitation, recent stu…

Cited by 0SourcecodeScholar
2023

GridMM: Grid Memory Map for Vision-and-Language Navigation

ICCV 2023poster

Vision-and-language navigation (VLN) enables the agent to navigate to a remote location following the natural language instruction in 3D environments. To represent the previously visited environment, most approaches for VLN implement memory using recurrent states, topological maps, or top-down seman…

Cited by 59PDFcodeScholar
2023

Guiding Pretraining in Reinforcement Learning with Large Language Models

ICML 2023poster

Reinforcement learning algorithms typically struggle in the absence of a dense, well-shaped reward function. Intrinsically motivated exploration methods address this limitation by rewarding agents for visiting novel states or transitions, but these methods offer limited benefits in large environment…

2023

KERM: Knowledge Enhanced Reasoning for Vision-and-Language Navigation

CVPR 2023poster

Vision-and-language navigation (VLN) is the task to enable an embodied agent to navigate to a remote location following the natural language instruction in real scenes. Most of the previous approaches utilize the entire features or object-centric features to represent navigable candidates. However,…

2023

Learning to Imagine: Distillation-Based Interactive Context Exploitation for Dialogue State Tracking

AAAI 2023technical

In dialogue state tracking (DST), the exploitation of dialogue history is a crucial research direction, and the existing DST models can be divided into two categories: full-history models and partial-history models. Since the “select first, use later” mechanism explicitly filters the distracting inf…

2023

Less than One-shot: Named Entity Recognition via Extremely Weak Supervision

EMNLP 2023long findings

We study the named entity recognition (NER) problem under the extremely weak supervision (XWS) setting, where only one example entity per type is given in a context-free way. While one can see that XWS is \emph{lighter than one-shot} in terms of the amount of supervision, we propose a novel method…

Cited by 0SourcecodeScholar
2023

Masked Imitation Learning: Discovering Environment-Invariant Modalities in Multimodal Demonstrations

IROS 2023poster

Multimodal demonstrations provide robots with an abundance of information to make sense of the world. However, such abundance may not always lead to good performance when it comes to learning sensorimotor control policies from human demonstrations. Extraneous data modalities can lead to state over-s…

Cited by 2SourceScholar
2023

Multi-Defendant Legal Judgment Prediction via Hierarchical Reasoning

EMNLP 2023long findings

Multiple defendants in a criminal fact description generally exhibit complex interactions, and cannot be well handled by existing Legal Judgment Prediction (LJP) methods which focus on predicting judgment results (e.g., law articles, charges, and terms of penalty) for single-defendant cases. To addr…

Cited by 0SourcecodeScholar
2023

PiMAE: Point Cloud and Image Interactive Masked Autoencoders for 3D Object Detection

CVPR 2023poster

Masked Autoencoders learn strong visual representations and achieve state-of-the-art results in several independent modalities, yet very few works have addressed their capabilities in multi-modality settings. In this work, we focus on point cloud and RGB image data, two modalities that are often pre…

2023

STEV: Stretchable Triboelectric E-skin enabled Proprioceptive Vibration Sensing for Soft Robot

ICRA 2023poster

Vibration perception is essential for robotic sensing and dynamic control. Nevertheless, due to the rigorous demand for sensor conformability and stretchability, enabling soft robots with proprioceptive vibration sensing remains challenging. This paper proposes a novel liquid metal-based stretchable…

Cited by 6SourceScholar
2023

ToxicChat: Unveiling Hidden Challenges of Toxicity Detection in Real-World User-AI Conversation

EMNLP 2023short findings

Despite remarkable advances that large language models have achieved in chatbots nowadays, maintaining a non-toxic user-AI interactive environment has become increasingly critical nowadays. However, previous efforts in toxicity detection have been mostly based on benchmarks derived from social media…

Cited by 0SourceScholar
2022

Beyond the Granularity: Multi-Perspective Dialogue Collaborative Selection for Dialogue State Tracking

ACL 2022long

In dialogue state tracking, dialogue history is a crucial material, and its utilization varies between different models. However, no matter how the dialogue history is used, each existing model uses its own consistent dialogue history during the entire state tracking process, regardless of which slo…

2022

Formulating Few-shot Fine-tuning Towards Language Model Pre-training: A Pilot Study on Named Entity Recognition

EMNLP 2022finding

Fine-tuning pre-trained language models is a common practice in building NLP models for various tasks, including the case with less supervision. We argue that under the few-shot setting, formulating fine-tuning closer to the pre-training objective shall be able to unleash more benefits from the pre-…

2022

HPT: Hierarchy-aware Prompt Tuning for Hierarchical Text Classification

EMNLP 2022main

Hierarchical text classification (HTC) is a challenging subtask of multi-label classification due to its complex label hierarchy.Recently, the pretrained language models (PLM)have been widely adopted in HTC through a fine-tuning paradigm. However, in this paradigm, there exists a huge gap between th…

2022

Incorporating Hierarchy into Text Encoder: a Contrastive Learning Approach for Hierarchical Text Classification

ACL 2022long

Hierarchical text classification is a challenging subtask of multi-label classification due to its complex label hierarchy. Existing methods encode text and label hierarchy separately and mix their representations for classification, where the hierarchy remains unchanged for all input text. Instead…

2022

Learning Adaptive Axis Attentions in Fine-tuning: Beyond Fixed Sparse Attention Patterns

ACL 2022findings

We present a comprehensive study of sparse attention patterns in Transformer models. We first question the need for pre-training with sparse attention and present experiments showing that an efficient fine-tuning only approach yields a slightly worse but still competitive model. Then we compare the…

Cited by 1SourcePDFScholar
2022

M$^4$I: Multi-modal Models Membership Inference

NeurIPS 2022accept

With the development of machine learning techniques, the attention of research has been moved from single-modal learning to multi-modal learning, as real-world data exist in the form of different modalities. However, multi-modal models often carry more information than single-modal models and they a…

2022

Rethinking the Setting of Semi-supervised Learning on Graphs

IJCAI 2022poster

We argue that the present setting of semisupervised learning on graphs may result in unfair comparisons, due to its potential risk of over-tuning hyper-parameters for models. In this paper, we highlight the significant influence of tuning hyper-parameters, which leverages the label information in th…

2021

Dual Slot Selector via Local Reliability Verification for Dialogue State Tracking

ACL 2021long

The goal of dialogue state tracking (DST) is to predict the current dialogue state given all previous dialogue contexts. Existing approaches generally predict the dialogue state at every turn from scratch. However, the overwhelming majority of the slots in each turn should simply inherit the slot va…

2021

“Average” Approximates “First Principal Component”? An Empirical Analysis on Representations from Neural Language Models

EMNLP 2021main

Contextualized representations based on neural language models have furthered the state of the art in various NLP tasks. Despite its great success, the nature of such representations remains a mystery. In this paper, we present an empirical property of these representations—”average” approximates “f…

2019

A Data-Efficient Framework for Training and Sim-to-Real Transfer of Navigation Policies

ICRA 2019poster

Learning effective visuomotor policies for robots purely from data is challenging, but also appealing since a learning-based system should not require manual tuning or calibration. In the case of a robot operating in a real environment the training process can be costly, time-consuming, and even dan…

Cited by 47SourceScholar