← Search

Jiayi Zhang

29 accepted papers

2026

AOrchestra: Automating Sub-Agent Creation for Agentic Orchestration

ICML 2026poster

Language agents have shown strong promise for task automation. Realizing this promise for increasingly complex, long-horizon tasks has driven the rise of a subagent-as-tools paradigm for multi-turn task solving. However, existing designs still lack a dynamic abstraction view of sub-agents, thereby h…

Cited by 0SourceScholar
2026

AutoWebWorld: Synthesizing Infinite Verifiable Web Environments via Finite State Machines

ICML 2026poster

The performance of autonomous Web GUI agents heavily relies on the quality and quantity of their training data. However, a fundamental bottleneck persists: collecting interaction trajectories from real-world websites is expensive and difficult to verify. The underlying state transitions are hidden, …

Cited by 0SourceScholar
2026

Dissecting Embodied Abilities in Multimodal Language Models through Skill-level Evaluation and Diagnosis

ICML 2026poster

Understanding the capability bottlenecks of embodied multimodal large language models (MLLMs) is crucial for improvement. However, existing embodied benchmarks fail to provide actionable insights because they focus on task-level evaluation rather than discovering capability bottlenecks. To address t…

Cited by 0SourceScholar
2026

InfoPO: Information-Driven Policy Optimization for User-Centric Agents

ICML 2026poster

Real-world user requests to LLM agents are often underspecified. Agents must interact to acquire missing information and make correct downstream decisions. However, current multi-turn GRPO-based methods often rely on trajectory-level reward computation, which leads to credit assignment problems and …

Cited by 0SourceScholar
2026

InteractComp: Evaluating Search Agents With Ambiguous Queries

ICML 2026poster

Language agents have demonstrated remarkable potential in web search and information retrieval. However, these search agents assume user queries are complete and unambiguous, an assumption that diverges from reality where users begin with incomplete queries requiring clarification through interactio…

Cited by 0SourceScholar
2026

Multi-modal Dynamic Proxy Learning for Personalized Multiple Clustering

AAAI 2026technical

Multiple clustering aims to discover diverse latent structures from different perspectives, yet existing methods generate exhaustive clusterings without discerning user interest, necessitating laborious manual screening. Current multi-modal solutions suffer from static semantic rigidity: predefined

Cited by 0SourcePDFScholar
2026

Verbalized Sampling: How to Mitigate Mode Collapse and Unlock LLM Diversity

ICML 2026poster

Post-training alignment often reduces LLM diversity, leading to a phenomenon known as mode collapse. Unlike prior work that attributes this effect to algorithmic limitations, we identify a fundamental, pervasive data-level driver: typicality bias in preference data, whereby annotators systematically…

Cited by 0SourceScholar
2026

VisJudge-Bench: Aesthetics and Quality Assessment of Visualizations

ICLR 2026poster

Visualization, a domain-specific yet widely used form of imagery, is an effective way to turn complex datasets into intuitive insights, and its value depends on whether data are faithfully represented, clearly communicated, and aesthetically designed. However, evaluating visualization quality is cha…

Cited by 0SourcecodeScholar
2025

AFlow: Automating Agentic Workflow Generation

ICLR 2025oral

Large language models (LLMs) have demonstrated remarkable potential in solving complex tasks across diverse domains, typically by employing agentic workflows that follow detailed instructions and operational sequences. However, constructing these workflows requires significant human effort, limiting…

2025

APHQ-ViT: Post-Training Quantization with Average Perturbation Hessian Based Reconstruction for Vision Transformers

CVPR 2025poster

Vision Transformers (ViTs) have become one of the most commonly used backbones for vision tasks. Despite their remarkable performance, they often suffer significant accuracy drop when quantized for practical deployment, particularly by post-training quantization (PTQ) under ultra-low bits. Recently,…

2025

Alpha-SQL: Zero-Shot Text-to-SQL using Monte Carlo Tree Search

ICML 2025poster

Text-to-SQL, which enables natural language interaction with databases, serves as a pivotal method across diverse industries. With new, more powerful large language models (LLMs) emerging every few months, fine-tuning has become incredibly costly, labor-intensive, and error-prone. As an alternative,…

Cited by 6SourcePDFScholar
2025

Atom of Thoughts for Markov LLM Test-Time Scaling

NeurIPS 2025poster

Large Language Models (LLMs) achieve superior performance through training-time scaling, and test-time scaling further enhances their capabilities by conducting effective reasoning during inference. However, as the scale of reasoning increases, existing test-time scaling methods suffer from accumul…

Cited by 0SourcecodeScholar
2025

Data Interpreter: An LLM Agent for Data Science

ACL 2025finding

Large Language Model (LLM)-based agents have excelled in various domains but face significant challenges when applied to data science workflows due to their complex, multi-stage nature. Current LLM-based agents struggle with non-linear relationships, recursive dependencies, implicit data- and logic-…

2025

Efficient Rectified Flow for Image Fusion

NeurIPS 2025poster

Image fusion is a fundamental and important task in computer vision, aiming to combine complementary information from different modalities to fuse images. In recent years, diffusion models have made significant developments in the field of image fusion. However, diffusion models often require comple…

Cited by 0SourceScholar
2025

FIMA-Q: Post-Training Quantization for Vision Transformers by Fisher Information Matrix Approximation

CVPR 2025highlight

Post-training quantization (PTQ) has stood out as a cost-effective and promising model compression approach over recent years, as it eliminates the need for retraining on the entire dataset. Unfortunately, most existing PTQ methods for Vision Transformers (ViTs) exhibit a notable drop in accuracy, e…

2025

Multi-Cali Anything: Dense Feature Multi-Frame Structure-from-Motion for Large-Scale Camera Array Calibration

IROS 2025

Calibrating large-scale camera arrays, such as those in dome-based setups, is time-intensive and typically requires dedicated captures of known patterns. While extrinsics in such arrays are fixed due to the physical setup, intrinsics often vary across sessions due to factors like lens adjustments or

Cited by 5SourcecodeScholar
2025

Self-Supervised Prompt Optimization

EMNLP 2025

Well-designed prompts are crucial for enhancing Large language models’ (LLMs) reasoning capabilities while aligning their outputs with task requirements across diverse domains. However, manually designed prompts require expertise and iterative experimentation. While existing prompt optimization meth

2025

Understanding How Value Neurons Shape the Generation of Specified Values in LLMs

EMNLP 2025

Rapid integration of large language models (LLMs) into societal applications has intensified concerns about their alignment with universal ethical principles, as their internal value representations remain opaque despite behavioral alignment advancements. Current approaches struggle to systematicall

Cited by 0SourcePDFScholar
2024

Analyzing the Role of Semantic Representations in the Era of Large Language Models

NAACL 2024long

Traditionally, natural language processing (NLP) models often use a rich set of features created by linguistic expertise, such as semantic representations. However, in the era of large language models (LLMs), more and more tasks are turned into generic, end-to-end sequence generation problems. In th…

2024

Generative Al-aided Joint Training-free Secure Semantic Communications via Multi-modal Prompts

ICASSP 2024accepted

Semantic communication (SemCom) holds promise for reducing network resource consumption while achieving the communications goal. However, the computational overheads in jointly training semantic encoders and decoders—and the subsequent deployment in network devices—are overlooked. Recent advances in…

Cited by 0SourceScholar
2023

Learning Audio-Visual Source Localization via False Negative Aware Contrastive Learning

CVPR 2023poster

Self-supervised audio-visual source localization aims to locate sound-source objects in video frames without extra annotations. Recent methods often approach this goal with the help of contrastive learning, which assumes only the audio and visual contents from the same video are positive samples for…

2023

Learning towards Selective Data Augmentation for Dialogue Generation

AAAI 2023technical

As it is cumbersome and expensive to acquire a huge amount of data for training neural dialog models, data augmentation is proposed to effectively utilize existing training samples. However, current data augmentation techniques on the dialog generation task mostly augment all cases in the training d…

Cited by 8SourcePDFScholar
2022

Audio—Visual Segmentation

ECCV 2022poster

"We propose to explore a new problem called audio-visual segmentation (AVS), in which the goal is to output a pixel-level map of the object(s) that produce sound at the time of the image frame. To facilitate this research, we construct the first audio-visual segmentation benchmark (AVSBench), provid…

2022

C3KG: A Chinese Commonsense Conversation Knowledge Graph

ACL 2022findings

Existing commonsense knowledge bases often organize tuples in an isolated manner, which is deficient for commonsense conversational models to plan the next steps. To fill the gap, we curate a large-scale multi-turn human-written conversation corpus, and create the first Chinese commonsense conversat…

2021

A Bi-Level Framework for Learning to Solve Combinatorial Optimization on Graphs

NeurIPS 2021poster

Combinatorial Optimization (CO) has been a long-standing challenging research topic featured by its NP-hard nature. Traditionally such problems are approximately solved with heuristic algorithms which are usually fast but may sacrifice the solution quality. Currently, machine learning for combinator…

2021

Reasoning in Dialog: Improving Response Generation by Context Reading Comprehension

AAAI 2021technical

In multi-turn dialog, utterances do not always take the full form of sentences (Carbonell 1983), which naturally makes understanding the dialog context more difficult. However, it is essential to fully grasp the dialog context to generate a reasonable response. Hence, in this paper, we propose to im…

2021

Writing Polishment with Simile: Task, Dataset and A Neural Approach

AAAI 2021technical

A simile is a figure of speech that directly makes a comparison, showing similarities between two different things, e.g. ``Reading papers can be dull sometimes,like watching grass grow". Human writers often interpolate appropriate similes into proper locations of the plain text to vivify their writi…