AIConfPaper
← Search

Hongxia Jin

55 accepted papers

2025

Better Exploiting Spatial Separability in Multichannel Speech Enhancement with an Align-and-Filter Network

Ching Hua Lee, Chouchang Yang, Yashas Malur Saidutta, Rakshith Sharma Srinivasa, Yilin Shen, Hongxia Jin
ICASSP 2025accepted

Multichannel speech enhancement (SE) techniques combine multiple microphone signals to extract clean speech from noisy mixtures based on spatial filtering. As the target speech may come from arbitrary, unknown directions, current deep learning-based SE systems could suffer from performance bottlenec…

Cited by 0Source ↗Scholar ↗
2025

FlexiGPT: Pruning and Extending Large Language Models with Low-Rank Weight Sharing

James Seale Smith, Chi-Heng Lin, Shikhar Tuli, Haris Jeelani, Shangqian Gao, Yilin Shen +2
NAACL 2025long

The rapid proliferation of large language models (LLMs) in natural language processing (NLP) has created a critical need for techniques that enable efficient deployment on memory-constrained devices without compromising performance. We present a method to prune LLMs that selectively prunes model blo…

Cited by 0Source ↗PDF ↗code ↗Scholar ↗
2025

MIB: Mixed Information Bottleneck for Out-of-Distribution Keyword Spotting

Yashas Malur Saidutta, Rakshith Sharma Srinivasa, Jaejin Cho, Ching Hua Lee, Chouchang Yang, Yilin Shen +1
ICASSP 2025accepted

Deep Keyword Spotting (KWS) systems continuously process audio streams to detect keywords. However, performance of deep neural networks degrade when the input data diverges from the training data; referred to as Out-of-Distribution (OOD) data problem. In this paper, we show performance degradation o…

Cited by 0Source ↗Scholar ↗
2025

MoDeGPT: Modular Decomposition for Large Language Model Compression

Chi-Heng Lin, Shangqian Gao, James Seale Smith, Abhishek Patel, Shikhar Tuli, Yilin Shen +2
ICLR 2025oral

Large Language Models (LLMs) have significantly advanced AI with their exceptional performance across a wide range of tasks. However, their extensive computational requirements restrict their use on devices with limited resources. While recent compression methods based on low-rank matrices show pote…

Cited by 7Source ↗PDF ↗Scholar ↗
2025

RestoreGrad: Signal Restoration Using Conditional Denoising Diffusion Models with Jointly Learned Prior

Ching-Hua Lee, Chouchang Yang, Jaejin Cho, Yashas Malur Saidutta, Rakshith Sharma Srinivasa, Yilin Shen +1
ICML 2025poster

Denoising diffusion probabilistic models (DDPMs) can be utilized to recover a clean signal from its degraded observation(s) by conditioning the model on the degraded signal. The degraded signals are themselves contaminated versions of the clean signals; due to this correlation, they may encompass ce…

Cited by 0Source ↗PDF ↗Scholar ↗
2024

Adaptive Rank Selections for Low-Rank Approximation of Language Models

Shangqian Gao, Ting Hua, Yen-Chang Hsu, Yilin Shen, Hongxia Jin
NAACL 2024long

Singular Value Decomposition (SVD) or its weighted variants has significantly progressed in compressing language models. Previous works assume the same importance for all operations and assign the same number of ranks for different layers in a language model. However, such a uniform rank selection i…

Cited by 6Source ↗PDF ↗code ↗Scholar ↗
2024

AlpaGasus: Training a Better Alpaca with Fewer Data

Lichang Chen, Shiyang Li, Jun Yan, Hai Wang, Kalpa Gunaratna, Vikas Yadav +5
ICLR 2024poster

Large language models~(LLMs) strengthen instruction-following capability through instruction-finetuning (IFT) on supervised instruction/response data. However, widely used IFT datasets (e.g., Alpaca's 52k data) surprisingly contain many low-quality instances with incorrect or irrelevant responses, w…

Cited by 235Source ↗PDF ↗code ↗Scholar ↗
2024

An MVDR-Embedded U-Net Beamformer for Effective and Robust Multichannel Speech Enhancement

Ching Hua Lee, Kashyap Patel, Chouchang Yang, Yilin Shen, Hongxia Jin
ICASSP 2024accepted

In multichannel speech enhancement (SE) systems, deep neural networks (DNNs) are often utilized to directly estimate the clean speech for effective beamforming. This approach, however, may not generalize adequately to new acoustic or noise conditions. Alternatively, DNNs can indirectly perform SE by…

Cited by 0Source ↗Scholar ↗
2024

Backdooring Instruction-Tuned Large Language Models with Virtual Prompt Injection

Jun Yan, Vikas Yadav, Shiyang Li, Lichang Chen, Zheng Tang, Hai Wang +3
NAACL 2024long

Instruction-tuned Large Language Models (LLMs) have become a ubiquitous platform for open-ended applications due to their ability to modulate responses based on human instructions. The widespread use of LLMs holds significant potential for shaping public perception, yet also risks being maliciously…

Cited by 109Source ↗PDF ↗code ↗Scholar ↗
2024

CIFD: Controlled Information Flow to Enhance Knowledge Distillation

Yashas Malur Saidutta, Rakshith Sharma Srinivasa, Jaejin Cho, Ching-Hua Lee, Chouchang Yang, Yilin Shen +1
NeurIPS 2024poster

Knowledge Distillation is the mechanism by which the insights gained from a larger teacher model are transferred to a smaller student model. However, the transfer suffers when the teacher model is significantly larger than the student. To overcome this, prior works have proposed training intermediat…

Cited by 2Source ↗PDF ↗Scholar ↗
2024

DISP-LLM: Dimension-Independent Structural Pruning for Large Language Models

Shangqian Gao, Chi-Heng Lin, Ting Hua, Zheng Tang, Yilin Shen, Hongxia Jin +1
NeurIPS 2024poster

Large Language Models (LLMs) have achieved remarkable success in various natural language processing tasks, including language modeling, understanding, and generation. However, the increased memory and computational costs associated with these models pose significant challenges for deployment on res…

Cited by 4Source ↗PDF ↗code ↗Scholar ↗
2024

DynaMo: Accelerating Language Model Inference with Dynamic Multi-Token Sampling

Shikhar Tuli, Chi-Heng Lin, Yen-Chang Hsu, Niraj Jha, Yilin Shen, Hongxia Jin
NAACL 2024long

Traditional language models operate autoregressively, i.e., they predict one token at a time. Rapid explosion in model sizes has resulted in high inference times. In this work, we propose DynaMo, a suite of multi-token prediction language models that reduce net inference times. Our models *dynamical…

Cited by 1Source ↗PDF ↗code ↗Scholar ↗
2024

Enabling Device Control Planning Capabilities of Small Language Model

Sudipta Paul, Lingyu Zhang, Yilin Shen, Hongxia Jin
ICASSP 2024accepted

Smart home device control is a difficult task if the instruction is abstract and the planner needs to adjust dynamic home configurations. With the increasing capability of Large Language Model (LLM), they have become the customary model for zero-shot planning tasks similar to smart home device contr…

Cited by 0Source ↗Scholar ↗
2024

End-To-End Personalized Cuff-Less Blood Pressure Monitoring Using ECG and PPG Signals

Suhas BN, Rakshith Sharma Srinivasa, Yashas Malur Saidutta, Jaejin Cho, Ching Hua Lee, Chouchang Yang +2
ICASSP 2024accepted

Cuffless blood pressure (BP) monitoring offers the potential for continuous, non-invasive healthcare but has been limited in adoption by existing models relying on handcrafted features from ECG and PPG signals. To overcome this, researchers have looked to deep learning. Along these lines, in this pa…

Cited by 0Source ↗Scholar ↗
2024

Explicit over Implict: Explicit Diversity Conditions for Effective Question Answer Generation

Vikas Yadav, Hyuk joon Kwon, Vijay Srinivasan, Hongxia Jin
COLING 2024main

Question Answer Generation (QAG) is an effective data augmentation technique to improve the accuracy of question answering systems, especially in low-resource domains. While recent pretrained and large language model-based QAG methods have made substantial progress, they face the critical issue of r…

Cited by 2Source ↗PDF ↗Scholar ↗
2024

Instruction-following Evaluation through Verbalizer Manipulation

Shiyang Li, Jun Yan, Hai Wang, Zheng Tang, Xiang Ren, Vijay Srinivasan +1
NAACL 2024findings

While instruction-tuned models have shown remarkable success in various natural language processing tasks, accurately evaluating their ability to follow instructions remains challenging. Existing benchmarks primarily focus on common instructions that align well with what the model learned during tra…

Cited by 27Source ↗PDF ↗code ↗Scholar ↗
2024

Leveraging Self-Supervised Speech Representations for Domain Adaptation in Speech Enhancement

Ching Hua Lee, Chouchang Yang, Rakshith Sharma Srinivasa, Yashas Malur Saidutta, Jaejin Cho, Yilin Shen +1
ICASSP 2024accepted

Deep learning based speech enhancement (SE) approaches could suffer from performance degradation due to mismatch between training and testing environments. A realistic situation is that an SE model trained on parallel noisy-clean utterances from one environment, the source domain, may fail to perfor…

Cited by 0Source ↗Scholar ↗
2024

SLiM: Speculative Decoding with Hypothesis Reduction

Chi-Heng Lin, Shikhar Tuli, James Smith, Yen-Chang Hsu, Yilin Shen, Hongxia Jin
NAACL 2024findings

Speculative decoding has emerged as a prominent alternative to autoregressive decoding for expediting inference in large language models (LLMs). However, prevailing assumptions often focus solely on latency reduction, neglecting the computational expenses. In this paper, we present Speculate Less, v…

Cited by 3Source ↗PDF ↗Scholar ↗
2024

Unleashing Multispectral Video's Potential in Semantic Segmentation: A Semi-supervised Viewpoint and New UAV-View Benchmark

Wei Ji, Jingjing Li, Wenbo Li, Yilin Shen, Li cheng, Hongxia Jin
NeurIPS 2024poster

Thanks to the rapid progress in RGB & thermal imaging, also known as multispectral imaging, the task of multispectral video semantic segmentation, or MVSS in short, has recently drawn significant attentions. Noticeably, it offers new opportunities in improving segmentation performance under unfavora…

Cited by 0Source ↗PDF ↗Scholar ↗
2024

Zero-Shot Intent Classification Using a Semantic Similarity Aware Contrastive Loss and Large Language Model

Jaejin Cho, Rakshith Sharma Srinivasa, Ching Hua Lee, Yashas Malur Saidutta, Chouchang Yang, Yilin Shen +1
ICASSP 2024accepted

Zero-shot systems can reduce the cost of collecting data and training in a new domain since they can work directly with the test data without further training. In this paper, we build zero-shot systems for intent classification, based on Semantic Similarity-aware Contrastive Loss (SSCL) that address…

Cited by 1Source ↗Scholar ↗
2023

CWCL: Cross-Modal Transfer with Continuously Weighted Contrastive Loss

Rakshith Sharma Srinivasa, Jaejin Cho, Chouchang Yang, Yashas Malur Saidutta, Ching-Hua Lee, Yilin Shen +1
NeurIPS 2023poster

This paper considers contrastive training for cross-modal 0-shot transfer wherein a pre-trained model in one modality is used for representation learning in another domain using pairwise data. The learnt models in the latter domain can then be used for a diverse set of tasks in a 0-shot way, similar…

Cited by 8Source ↗PDF ↗Scholar ↗
2023

Dynamic Low-rank Estimation for Transformer-based Language Models

Ting Hua, Xiao Li, Shangqian Gao, Yen-Chang Hsu, Yilin Shen, Hongxia Jin
EMNLP 2023long findings

Matrix decomposition methods, such as Singular Value Decomposition (SVD) and its importance-weighted variants, have been widely used for compressing Transformer-based language models. While importance-weighted decomposition methods alleviate the strong assumption of equal importance for each paramet…

Cited by 0Source ↗Scholar ↗
2023

ESC: Exploration with Soft Commonsense Constraints for Zero-shot Object Navigation

Kaiwen Zhou, Kaizhi Zheng, Connor Pryor, Yilin Shen, Hongxia Jin, Lise Getoor +1
ICML 2023poster

The ability to accurately locate and navigate to a specific object is a crucial capability for embodied agents that operate in the real world and interact with objects to complete tasks. Such object navigation tasks usually require large-scale training in visual environments with labeled objects, wh…

Cited by 115Source ↗PDF ↗Scholar ↗
2023

GOHSP: A Unified Framework of Graph and Optimization-Based Heterogeneous Structured Pruning for Vision Transformer

Miao Yin, Burak Uzkent, Yilin Shen, Hongxia Jin, Bo Yuan
AAAI 2023technical

The recently proposed Vision transformers (ViTs) have shown very impressive empirical performance in various computer vision tasks, and they are viewed as an important type of foundation model. However, ViTs are typically constructed with large-scale sizes, which then severely hinder their potential…

Cited by 22Source ↗PDF ↗Scholar ↗
2023

Improved Mask-Based Neural Beamforming for Multichannel Speech Enhancement by Snapshot Matching Masking

Ching Hua Lee, Chouchang Yang, Yilin Shen, Hongxia Jin
ICASSP 2023accepted

In multichannel speech enhancement (SE), time-frequency (T-F) mask-based neural beamforming algorithms take advantage of deep neural networks to predict T-F masks that represent speech and noise dominance. The predicted masks are subsequently leveraged to estimate the speech and noise power spectral…

Cited by 0Source ↗Scholar ↗
2023

Learning to Jointly Share and Prune Weights for Grounding Based Vision and Language Models

Shangqian Gao, Burak Uzkent, Yilin Shen, Heng Huang, Hongxia Jin
ICLR 2023poster

Transformers have seen growing interest in processing different modalities, including language and image data. As a result, we can process vision and language data using transformers that are architecturally similar. Leveraging this feature of transformers, we propose weight sharing across two tran…

Cited by 10Source ↗PDF ↗Scholar ↗
2023

To Wake-Up or Not to Wake-Up: Reducing Keyword False Alarm by Successive Refinement

Yashas Malur Saidutta, Rakshith Sharma Srinivasa, Ching Hua Lee, Chouchang Yang, Yilin Shen, Hongxia Jin
ICASSP 2023accepted

Keyword spotting systems continuously process audio streams to detect keywords. One of the most challenging tasks in designing such systems is to reduce False Alarm (FA) which happens when the system falsely registers a keyword despite the keyword not being uttered. In this paper, we propose a simpl…

Cited by 0Source ↗Scholar ↗
2022

Explainable Slot Type Attentions to Improve Joint Intent Detection and Slot Filling

Kalpa Gunaratna, Vijay Srinivasan, Akhila Yerukola, Hongxia Jin
EMNLP 2022finding

Joint intent detection and slot filling is a key research topic in natural language understanding (NLU). Existing joint intent and slot filling systems analyze and compute features collectively for all slot types, and importantly, have no way to explain the slot filling model decisions. In this work…

Cited by 6Source ↗PDF ↗Scholar ↗
2022

ISEEQ: Information Seeking Question Generation Using Dynamic Meta-Information Retrieval and Knowledge Graphs

Manas Gaur, Kalpa Gunaratna, Vijay Srinivasan, Hongxia Jin
AAAI 2022technical

Conversational Information Seeking (CIS) is a relatively new research area within conversational AI that attempts to seek information from end-users in order to understand and satisfy the users' needs. If realized, such a CIS system has far-reaching benefits in the real world; for example, CIS syste…

Cited by 64Source ↗PDF ↗code ↗Scholar ↗
2022

Improving Zero-Shot Phrase Grounding via Reasoning on External Knowledge and Spatial Relations

Zhan Shi, Yilin Shen, Hongxia Jin, Xiaodan Zhu
AAAI 2022technical

Phrase grounding is a multi-modal problem that localizes a particular noun phrase in an image referred to by a text query. In the challenging zero-shot phrase grounding setting, the existing state-of-the-art grounding models have limited capacity in handling the unseen phrases. Humans, however, can…

Cited by 9Source ↗PDF ↗Scholar ↗
2022

Language model compression with weighted low-rank factorization

Yen-Chang Hsu, Ting Hua, Sungen Chang, Qian Lou, Yilin Shen, Hongxia Jin
ICLR 2022poster

Factorizing a large matrix into small matrices is a popular strategy for model compression. Singular value decomposition (SVD) plays a vital role in this compression strategy, approximating a learned matrix with fewer parameters. However, SVD minimizes the squared error toward reconstructing the ori…

Cited by 113Source ↗PDF ↗Scholar ↗
2022

Numerical Optimizations for Weighted Low-rank Estimation on Language Models

Ting Hua, Yen-Chang Hsu, Felicity Wang, Qian Lou, Yilin Shen, Hongxia Jin
EMNLP 2022main

Singular value decomposition (SVD) is one of the most popular compression methods that approximate a target matrix with smaller matrices. However, standard SVD treats the parameters within the matrix with equal importance, which is a simple but unrealistic assumption. The parameters of a trained neu…

Cited by 14Source ↗PDF ↗Scholar ↗
2022

Text-Based Interactive Recommendation via Offline Reinforcement Learning

Ruiyi Zhang, Tong Yu, Yilin Shen, Hongxia Jin
AAAI 2022technical

Interactive recommendation with natural-language feedback can provide richer user feedback and has demonstrated advantages over traditional recommender systems. However, the classical online paradigm involves iteratively collecting experience via interaction with users, which is expensive and risky.…

Cited by 15Source ↗PDF ↗Scholar ↗
2021

Always Be Dreaming: A New Approach for Data-Free Class-Incremental Learning

James Smith, Yen-Chang Hsu, Jonathan Balloch, Yilin Shen, Hongxia Jin, Zsolt Kira
ICCV 2021poster

Modern computer vision applications suffer from catastrophic forgetting when incrementally learning new concepts over time. The most successful approaches to alleviate this forgetting require extensive replay of previously seen data, which is problematic when memory constraints or data legality conc…

Cited by 219PDF ↗code ↗Scholar ↗
2021

Automatic Mixed-Precision Quantization Search of BERT

Changsheng Zhao, Ting Hua, Yilin Shen, Qian Lou, Hongxia Jin
IJCAI 2021poster

Pre-trained language models such as BERT have shown remarkable effectiveness in various natural language processing tasks. However, these models usually contain millions of parameters, which prevent them from the practical deployment on resource-constrained devices. Knowledge distillation, Weight pr…

Cited by 22Source ↗PDF ↗code ↗Scholar ↗
2021

Enhancing the generalization for Intent Classification and Out-of-Domain Detection in SLU

Yilin Shen, Yen-Chang Hsu, Avik Ray, Hongxia Jin
ACL 2021long

Intent classification is a major task in spoken language understanding (SLU). Since most models are built with pre-collected in-domain (IND) training utterances, their ability to detect unsupported out-of-domain (OOD) utterances has a critical effect in practical use. Recent works have shown that us…

Cited by 45Source ↗PDF ↗Scholar ↗
2021

Hyperparameter-free Continuous Learning for Domain Classification in Natural Language Understanding

Ting Hua, Yilin Shen, Changsheng Zhao, Yen-Chang Hsu, Hongxia Jin
NAACL 2021long

Domain classification is the fundamental task in natural language understanding (NLU), which often requires fast accommodation to new emerging domains. This constraint makes it impossible to retrain all previous domains, even if they are accessible to the new model. Most existing continual learning…

Cited by 5Source ↗PDF ↗code ↗Scholar ↗
2021

Negative Data Augmentation

Abhishek Sinha, Kumar Ayush, Jiaming Song, Burak Uzkent, Hongxia Jin, Stefano Ermon
ICLR 2021poster

Data augmentation is often used to enlarge datasets with synthetic samples generated in accordance with the underlying data distribution. To enable a wider range of augmentations, we explore negative data augmentation strategies (NDA) that intentionally create out-of-distribution samples. We show th…

Cited by 102Source ↗PDF ↗code ↗Scholar ↗
2020

Conditional Image Repainting via Semantic Bridge and Piecewise Value Function

Shuchen Weng, Wenbo Li, Dawei Li, Hongxia Jin, Boxin Shi
ECCV 2020poster

We study conditional image repainting where a model is trained to generate visual content conditioned on user inputs, and composite the generated content seamlessly onto a user provided image while preserving the semantics of users' inputs. The content generation community have been pursuing to lowe…

Cited by 6Source ↗PDF ↗Scholar ↗
2020

Explainable and Efficient Sequential Correlation Network for 3D Single Person Concurrent Activity Detection

Yi Wei, Wenbo Li, Ming-Ching Chang, Hongxia Jin, Siwei Lyu
IROS 2020poster

We present the sequential correlation network (SCN) to improve concurrent activity detection. SCN combines a recurrent neural network and a correlation model hierarchically to model the complex correlations and temporal dynamics of concurrent activities. SCN has several advantages that enable effect…

Cited by 2Source ↗Scholar ↗
2020

Generalized ODIN: Detecting Out-of-Distribution Image Without Learning From Out-of-Distribution Data

Yen-Chang Hsu, Yilin Shen, Hongxia Jin, Zsolt Kira
CVPR 2020poster

Deep neural networks have attained remarkable performance when applied to data that comes from the same distribution as that of the training set, but can significantly degrade otherwise. Therefore, detecting whether an example is out-of-distribution (OoD) is crucial to enable a system that can rejec…

Cited by 725PDF ↗Scholar ↗
2020

MISC: Multi-Condition Injection and Spatially-Adaptive Compositing for Conditional Person Image Synthesis

Shuchen Weng, Wenbo Li, Dawei Li, Hongxia Jin, Boxin Shi
CVPR 2020poster

In this paper, we explore synthesizing person images with multiple conditions for various backgrounds. To this end, we propose a framework named "MISC" for conditional image generation and image compositing. For conditional image generation, we improve the existing condition injection mechanisms by…

Cited by 37PDF ↗Scholar ↗
2019

Adversarial Multi-label Prediction for Spoken and Visual Signal Tagging

Yue Deng, KaWai Chen, Yilin Shen, Hongxia Jin
ICASSP 2019accepted

We introduce an adversarial multi-label classification (ADMLC) framework to improve the robustness and performance of existing algorithms on multi-domain signals. The core contribution of our ADMLC is the innovation of an `adversarial module' that serves as a critic to provide augmenting information…

Cited by 0Source ↗Scholar ↗
2019

SLiQA-I: Towards Cold-start Development of End-to-end Spoken Language Interface for Question Answering

Yilin Shen, Yu Wang, Abhishek Patel, Hongxia Jin
ICASSP 2019accepted

Question answering (QA) has become a key capability for voice enabled personal assistants to automatically answer various user questions. However, the development of a spoken language interface for QA in a new domain is time consuming and requires a lot of human labors. Thus, it is crucially desirab…

Cited by 0Source ↗Scholar ↗
2019

Taking a HINT: Leveraging Explanations to Make Vision and Language Models More Grounded

Ramprasaath R. Selvaraju, Stefan Lee, Yilin Shen, Hongxia Jin, Shalini Ghosh, Larry Heck +2
ICCV 2019poster

Many vision and language models suffer from poor visual grounding -- often falling back on easy-to-learn language priors rather than basing their decisions on visual concepts in the image. In this work, we propose a generic approach called Human Importance-aware Network Tuning (HINT) that effectivel…

Cited by 305PDF ↗Scholar ↗
2019

Text-Based Interactive Recommendation via Constraint-Augmented Reinforcement Learning

Ruiyi Zhang, Tong Yu, Yilin Shen, Hongxia Jin, Changyou Chen
NeurIPS 2019accepted

Text-based interactive recommendation provides richer user preferences and has demonstrated advantages over traditional interactive recommender systems. However, recommendations can easily violate preferences of users from their past natural-language feedback, since the recommender needs to explore…

Cited by 155Source ↗PDF ↗code ↗Scholar ↗
2016

Efficient Private Empirical Risk Minimization for High-dimensional Learning

Shiva Prasad Kasiviswanathan, Hongxia Jin
ICML 2016poster

Dimensionality reduction is a popular approach for dealing with high dimensional data that leads to substantial computational savings. Random projections are a simple and effective method for universal dimensionality reduction with rigorous theoretical guarantees. In this paper, we theoretically stu…

Cited by 85Source ↗PDF ↗Scholar ↗