← Search

Yu Shi

27 accepted papers

2026

Degradation-Robust Fusion: An Efficient Degradation-Aware Diffusion Framework for Multimodal Image Fusion in Arbitrary Degradation Scenarios

CVPR 2026

Complex degradations like noise, blur, and low resolution are typical challenges in real-world image fusion tasks, limiting the performance and practicality of existing methods. End-to-end neural network-based approaches are generally simple to design and highly efficient in inference, but their bla

Cited by 0SourcecodeScholar
2026

FlexProtein: Joint Sequence and Structure Pretraining for Protein Modeling

ICLR 2026poster

Protein foundation models have advanced rapidly, with most approaches falling into two dominant paradigms. Sequence-only language models (e.g., ESM-2) capture sequence semantics at scale but lack structural grounding. MSA-based predictors (e.g., AlphaFold 2/3) achieve accurate folding by exploiting…

Cited by 0SourceScholar
2026

Kronos: A Foundation Model for the Language of Financial Markets

AAAI 2026technical

The success of large-scale pre-training paradigm, exemplified by Large Language Models (LLMs), has inspired the development of Time Series Foundation Models (TSFMs). However, their application to financial candlestick (K-line) data remains limited, often underperforming non-pre-trained architectures

Cited by 0SourcePDFScholar
2026

Navigating the Alpha Jungle: An LLM-Powered MCTS Framework for Formulaic Alpha Factor Mining

AAAI 2026technical

Alpha factor mining is pivotal in quantitative investment for identifying predictive signals from complex financial data. While traditional formulaic alpha mining relies on human expertise, contemporary automated methods, such as those based on genetic programming or reinforcement learning, often st

Cited by 0SourcePDFScholar
2026

TumorChain: Interleaved Multimodal Chain-of-Thought Reasoning for Traceable Clinical Tumor Analysis

ICLR 2026poster

Accurate tumor analysis is central to clinical radiology and precision oncology, where early detection, reliable lesion characterization, and pathology-level risk assessment directly guide diagnosis, staging, and treatment planning. Chain-of-Thought (CoT) reasoning is particularly critical in this s…

Cited by 0SourcecodeScholar
2026

UM-Text: A Unified Multimodal Model for Image Understanding and Visual Text Editing

AAAI 2026technical

With the rapid advancement of image generation, visual text editing using natural language instructions has received increasing attention. The main challenge of this task is to fully understand the instruction and reference image, and thus generate visual text that is style-consistent with the imag

Cited by 0SourcePDFScholar
2025

E2Former: An Efficient and Equivariant Transformer with Linear-Scaling Tensor Products

NeurIPS 2025spotlight

Equivariant Graph Neural Networks (EGNNs) have demonstrated significant success in modeling microscale systems, including those in chemistry, biology and materials science. However, EGNNs face substantial computational challenges due to the high cost of constructing edge features via spherical tenso…

Cited by 0SourceScholar
2024

Actions Speak Louder than Words: Trillion-Parameter Sequential Transducers for Generative Recommendations

ICML 2024poster

Large-scale recommendation systems are characterized by their reliance on high cardinality, heterogeneous features and the need to handle tens of billions of user actions on a daily basis. Despite being trained on huge volume of data with thousands of features, most Deep Learning Recommendation Mode…

2024

Deep Semantic Graph Transformer for Multi-View 3D Human Pose Estimation

AAAI 2024technical

Most Graph Convolutional Networks based 3D human pose estimation (HPE) methods were involved in single-view 3D HPE and utilized certain spatial graphs, existing key problems such as depth ambiguity, insufficient feature representation, or limited receptive fields. To address these issues, we propose…

2024

Modality-Agnostic Structural Image Representation Learning for Deformable Multi-Modality Medical Image Registration

CVPR 2024highlight

Establishing dense anatomical correspondence across distinct imaging modalities is a foundational yet challenging procedure for numerous medical image analysis studies and image-guided radiotherapy. Existing multi-modality image registration algorithms rely on statistical-based similarity measures o…

Cited by 12SourcePDFScholar
2024

Physical Consistency Bridges Heterogeneous Data in Molecular Multi-Task Learning

NeurIPS 2024poster

In recent years, machine learning has demonstrated impressive capability in handling molecular science tasks. To support various molecular properties at scale, machine learning models are trained in the multi-task learning paradigm. Nevertheless, data of different molecular properties are often not…

Cited by 1SourcePDFScholar
2024

i-Code V2: An Autoregressive Generation Framework over Vision, Language, and Speech Data

NAACL 2024findings

The convergence of text, visual, and audio data is crucial towards human-like artificial intelligence, however the current Vision-Language-Speech landscape is dominated by encoder-only models that lack generative abilities. We propose closing this gap with i-Code V2, one of the first models capable…

Cited by 3SourcePDFScholar
2023

CancerUniT: Towards a Single Unified Model for Effective Detection, Segmentation, and Diagnosis of Eight Major Cancers Using a Large Collection of CT Scans

ICCV 2023poster

Human readers or radiologists routinely perform full-body multi-organ multi-disease detection and diagnosis in clinical practice, while most medical AI systems are built to focus on single organs with a narrow list of a few diseases. This might severely limit AI's clinical adoption. A certain number…

Cited by 12PDFScholar
2023

Code-Switching Text Generation and Injection in Mandarin-English ASR

ICASSP 2023accepted

Code-switching speech refers to a means of expression by mixing two or more languages within a single utterance. Automatic Speech Recognition (ASR) with End-to-End (E2E) modeling for such speech can be a challenging task due to the lack of data. In this study, we investigate text generation and inje…

Cited by 0SourceScholar
2023

Devil Is in the Queries: Advancing Mask Transformers for Real-World Medical Image Segmentation and Out-of-Distribution Localization

CVPR 2023highlight

Real-world medical image segmentation has tremendous long-tailed complexity of objects, among which tail conditions correlate with relatively rare diseases and are clinically significant. A trustworthy medical AI algorithm should demonstrate its effectiveness on tail conditions to avoid clinically d…

Cited by 28SourcePDFScholar
2023

Efficient End-to-End Video Question Answering with Pyramidal Multimodal Transformer

AAAI 2023technical

This paper presents a new method for end-to-end Video Question Answering (VideoQA), aside from the current popularity of using large-scale pre-training with huge feature extractors. We achieve this with a pyramidal multimodal transformer (PMT) model, which simply incorporates a learnable word embedd…

2023

Z-Code++: A Pre-trained Language Model Optimized for Abstractive Summarization

ACL 2023long

This paper presents Z-Code++, a new pre-trained language model optimized for abstractive text summarization. The model extends the state-of-the-art encoder-decoder model using three techniques. First, we use a two-phase pre-training to improve the model’s performance on low-resource summarization ta…

2023

i-Code: An Integrative and Composable Multimodal Learning Framework

AAAI 2023technical

Human intelligence is multimodal; we integrate visual, linguistic, and acoustic signals to maintain a holistic worldview. Most current pretraining methods, however, are limited to one or two modalities. We present i-Code, a self-supervised pretraining framework where users may flexibly combine the m…

2022

Multilevel Hierarchical Network with Multiscale Sampling for Video Question Answering

IJCAI 2022poster

Video question answering (VideoQA) is challenging given its multimodal combination of visual understanding and natural language processing. While most existing approaches ignore the visual appearance-motion information at different temporal scales, it is unknown how to incorporate the multilevel pro…

2022

Optimizing Alignment of Speech and Language Latent Spaces for End-To-End Speech Recognition and Understanding

ICASSP 2022accepted

The advances in attention-based encoder-decoder (AED) networks have brought great progress to end-to-end (E2E) automatic speech recognition (ASR). One way to further improve the performance of AED-based E2E ASR is to introduce an extra text encoder for leveraging extensive text data and thus capture…

Cited by 0SourceScholar
2022

Quantized Training of Gradient Boosting Decision Trees

NeurIPS 2022accept

Recent years have witnessed significant success in Gradient Boosting Decision Trees (GBDT) for a wide range of machine learning applications. Generally, a consensus about GBDT's training algorithms is gradients and statistics are computed based on high-precision floating points. In this paper, we in…

Cited by 31SourcePDFScholar
2021

Generating Human Readable Transcript for Automatic Speech Recognition with Pre-Trained Language Model

ICASSP 2021accepted

Modern Automatic Speech Recognition (ASR) systems can achieve high performance in terms of recognition accuracy. However, a perfectly accurate transcript still can be challenging to read due to disfluency, filter words, and other errata common in spoken communication. Many downstream tasks and human…

Cited by 0SourceScholar
2021

Speech-Language Pre-Training for End-to-End Spoken Language Understanding

ICASSP 2021accepted

End-to-end (E2E) spoken language understanding (SLU) can infer semantics directly from speech signal without cascading an automatic speech recognizer (ASR) with a natural language understanding (NLU) module. However, paired utterance recordings and corresponding semantics may not always be available…

Cited by 0SourceScholar