← Search

Hai Jin

48 accepted papers

2026

APT: Towards Universal Scene Graph Generation via Plug-in Adaptive Prompt Tuning

ICLR 2026poster

Scene Graph Generation (SGG) is pivotal for structured visual understanding, yet it remains hindered by a fundamental limitation: the reliance on fixed, frozen semantic representations from pre-trained language models. These semantic priors, while beneficial in other domains, are inherently misalign…

Cited by 0SourcecodeScholar
2026

EnViT: Enhancing the Performance of Early-Exit Vision Transformers via Exit-Aware Structured Dropout-Enabled Self-Distillation

AAAI 2026technical

Vision Transformers (ViTs) have gained significant attention and widespread adoption due to their impressive performance in various computer vision tasks. However, in practice, their substantial computational overhead often leads to high inference latency and increased overheads when deployed on res

Cited by 0SourcePDFScholar
2026

Figma2Code: Automating Multimodal Design to Code in the Wild

ICLR 2026poster

Front-end development constitutes a substantial portion of software engineering, yet converting design mockups into production-ready *User Interface* (UI) code remains tedious and time-costly. While recent work has explored automating this process with *Multimodal Large Language Models* (MLLMs), e…

Cited by 0SourceScholar
2026

Neuromem: A Granular Decomposition of the Streaming Lifecycle in External Memory for LLMs

ICML 2026poster

Most evaluations of External Memory Module assume a static setting: memory is built offline and queried at a fixed state. In practice, memory is streaming: new facts arrive continuously, insertions interleave with retrievals, and the memory state evolves while the model is serving queries. In this r…

Cited by 0SourceScholar
2026

NuWa: Deriving Lightweight Class-Specific Vision Transformers for Edge Devices

CVPR 2026

Vision Transformers (ViTs) often need to be compressed for deployment on resource-constrained edge devices like drones and smart vehicles. However, existing model compression methods ignore that many edge devices only require the knowledge of specific classes for their applications. As a result, the

Cited by 0SourcecodeScholar
2026

RLAP-CLIP: Continual Multimodal Learning with Prototype Adaptation and Difficulty-Aware Routing

ICLR 2026poster

Vision-language models, such as CLIP, achieve strong zero-shot performance through contrastive pre-training but face significant challenges in class-incremental image classification scenarios. When learning new tasks sequentially, current methods suffer from degradation in prototype quality due to p…

Cited by 0SourceScholar
2026

SAGE: A Dataflow-Native Framework for Modular, Controllable, and Transparent LLM-Augmented Reasoning

ICML 2026poster

LLM applications increasingly execute as end-to-end inference pipelines that couple generation with retrieval, stateful memory, context refinement, and tool use under strict tail-latency and SLO constraints. Today, these stages are often stitched together as RPC-connected services, obscuring cross-s…

Cited by 0SourceScholar
2026

Unleashing the Intrinsic Visual Representation Capability of Multimodal Large Language Models

CVPR 2026

Multimodal Large Language Models (MLLMs) have demonstrated remarkable proficiency in multimodal tasks.Despite their impressive performance, MLLMs suffer from the modality imbalance issue, where visual information is often underutilized compared to textual representations in deeper layers, leading to

Cited by 0SourcecodeScholar
2026

Vulcan: Crafting Compact Class-Specific Vision Transformers For Edge Intelligence

ICLR 2026poster

Large Vision Transformers (ViTs) must often be compressed before they can be deployed on resource-constrained edge devices. However, many edge devices require only part of the *all-classes* knowledge of a pre-trained ViT in their corresponding application scenarios. This is overlooked by existing c…

Cited by 0SourcecodeScholar
2025

CodeSync: Synchronizing Large Language Models with Dynamic Code Evolution at Scale

ICML 2025poster

Large Language Models (LLMs) have exhibited exceptional performance in software engineering yet face challenges in adapting to continually evolving code knowledge, particularly the frequent updates of third-party library APIs. This limitation, rooted in the static pre-training datasets, often result…

2025

Dataflow-Guided Neuro-Symbolic Language Models for Type Inference

ICML 2025poster

Language Models (LMs) are increasingly used for type inference, aiding in error detection and software development. Some real-world deployments of LMs require the model to run on local machines to safeguard the intellectual property of the source code. This setting often limits the size of the LMs…

Cited by 0SourcePDFScholar
2025

Multi-Turn Jailbreaking Large Language Models via Attention Shifting

AAAI 2025technical

Large Language Models (LLMs) have achieved significant performance in various natural language processing tasks but also pose safety and ethical threats, thus requiring red teaming and alignment processes to bolster their safety. To effectively exploit these aligned LLMs, recent studies have introdu…

Cited by 0SourcePDFScholar
2025

NaFV-Net: An Adversarial Four-view Network for Mammogram Classification

AAAI 2025technical

Breast cancer remains a leading cause of mortality among women, with millions of new cases diagnosed annually. Early detection through screening is crucial. Using neural networks to improve the accuracy of breast cancer screening has become increasingly important. In accordance with radiologists' pr…

2025

NumbOD: A Spatial-Frequency Fusion Attack Against Object Detectors

AAAI 2025technical

With the advancement of deep learning, object detectors (ODs) with various architectures have achieved significant success in complex scenarios like autonomous driving. Previous adversarial attacks against ODs have been focused on designing customized attacks targeting their specific structures (eg,…

2025

Sign2Vis: Automated Data Visualization from Sign Language

ACL 2025finding

Data visualizations, such as bar charts and histograms, are essential for analyzing and exploring data, enabling the effective communication of insights. While existing methods have been proposed to translate natural language descriptions into visualization queries, they focus solely on spoken langu…

2025

Sim-LLM: Optimizing LLM Inference at the Edge through Inter-Task KV Reuse

NeurIPS 2025poster

KV cache technology, by storing key-value pairs, helps reduce the computational overhead incurred by *large language models* (LLMs). It facilitates their deployment on resource-constrained edge computing nodes like edge servers. However, as the complexity and size of tasks increase, KV cache usage l…

Cited by 0SourcecodeScholar
2025

Vanish into Thin Air: Cross-prompt Universal Adversarial Attacks for SAM2

NeurIPS 2025spotlight

Recent studies reveal the vulnerability of the image segmentation foundation model SAM to adversarial examples. Its successor, SAM2, has attracted significant attention due to its strong generalization capability in video segmentation. However, its robustness remains unexplored, and it is unclear wh…

Cited by 0SourceScholar
2024

An Efficient Subgraph-Inferring Framework for Large-Scale Heterogeneous Graphs

AAAI 2024technical

Heterogeneous Graph Neural Networks (HGNNs) play a vital role in advancing the field of graph representation learning by addressing the complexities arising from diverse data types and interconnected relationships in real-world scenarios. However, traditional HGNNs face challenges when applied to la…

2024

Are Your Models Still Fair? Fairness Attacks on Graph Neural Networks via Node Injections

NeurIPS 2024poster

Despite the remarkable capabilities demonstrated by Graph Neural Networks (GNNs) in graph-related tasks, recent research has revealed the fairness vulnerabilities in GNNs when facing malicious adversarial attacks. However, all existing fairness attacks require manipulating the connectivity between e…

2024

CausalNET: Unveiling Causal Structures on Event Sequences by Topology-Informed Causal Attention

IJCAI 2024poster

Causal discovery on event sequences holds a pivotal significance across domains such as healthcare, finance, and industrial systems. The crux of this endeavor lies in unraveling causal structures among event types, typically portrayed as directed acyclic graphs (DAGs). Nonetheless, prevailing method…

2024

Correcting Pronoun Homophones with Subtle Semantics in Chinese Speech Recognition

COLING 2024main

Speech recognition is becoming prevalent in daily life. However, due to the similar semantic context of the entities and the overlap of Chinese pronunciation, the pronoun homophone, especially “他/她/它 (he/she/it)”, (their pronunciation is “Tā”) is usually recognized incorrectly. It poses a challenge…

2024

DarkSAM: Fooling Segment Anything Model to Segment Nothing

NeurIPS 2024poster

Segment Anything Model (SAM) has recently gained much attention for its outstanding generalization to unseen data and tasks. Despite its promising prospect, the vulnerabilities of SAM, especially to universal adversarial perturbation (UAP) have not been thoroughly investigated yet. In this paper, we…

2024

Generalization-Enhanced Code Vulnerability Detection via Multi-Task Instruction Fine-Tuning

ACL 2024findings

Code Pre-trained Models (CodePTMs) based vulnerability detection have achieved promising results over recent years. However, these models struggle to generalize as they typically learn superficial mapping from source code to labels instead of understanding the root causes of code vulnerabilities, re…

2024

Harnessing the Power of Large Language Model for Uncertainty Aware Graph Processing

COLING 2024main

Handling graph data is one of the most difficult tasks. Traditional techniques, such as those based on geometry and matrix factorization, rely on assumptions about the data relations that become inadequate when handling large and complex graph data. On the other hand, deep learning approaches demons…

2024

Intersecting-Boundary-Sensitive Fingerprinting for Tampering Detection of DNN Models

ICML 2024poster

Cloud-based AI services offer numerous benefits but also introduce vulnerabilities, allowing for tampering with deployed DNN models, ranging from injecting malicious behaviors to reducing computing resources. Fingerprint samples are generated to query models to detect such tampering. In this paper,…

2024

Iterative Refinement of Project-Level Code Context for Precise Code Generation with Compiler Feedback

ACL 2024findings

Large Language Models (LLMs) have shown remarkable progress in automated code generation. Yet, LLM-generated code may contain errors in API usage, class, data structure, or missing project-specific information. As much of this project-specific context cannot fit into the prompts of LLMs, we must fin…

2024

MISA: Unveiling the Vulnerabilities in Split Federated Learning

ICASSP 2024accepted

Federated learning (FL) and split learning (SL) are prevailing distributed paradigms in recent years. They both enable shared global model training while keeping data localized on users’ devices. The former excels in parallel execution capabilities, while the latter enjoys low dependence on edge com…

Cited by 0SourceScholar
2024

Unlearnable 3D Point Clouds: Class-wise Transformation Is All You Need

NeurIPS 2024poster

Traditional unlearnable strategies have been proposed to prevent unauthorized users from training on the 2D image data. With more 3D point cloud data containing sensitivity information, unauthorized usage of this new type data has also become a serious concern. To address this, we propose the first…

2023

A Composite Multi-Attention Framework for Intraoperative Hypotension Early Warning

AAAI 2023technical

Intraoperative hypotension (IOH) events warning plays a crucial role in preventing postoperative complications, such as postoperative delirium and mortality. Despite significant efforts, two fundamental problems limit its wide clinical use. The well-established IOH event warning systems are often bu…

Cited by 5SourcePDFScholar
2023

AsT: An Asymmetric-Sensitive Transformer for Osteonecrosis of the Femoral Head Detection (Student Abstract)

AAAI 2023technical

Early diagnosis of osteonecrosis of the femoral head (ONFH) can inhibit the progression and improve femoral head preservation. The radiograph difference between early ONFH and healthy ones is not apparent to the naked eye. It is also hard to produce a large dataset to train the classification model.…

Cited by 0SourcePDFScholar
2023

Cross-links Matter for Link Prediction: Rethinking the Debiased GNN from a Data Perspective

NeurIPS 2023poster

Recently, the bias-related issues in GNN-based link prediction have raised widely spread concerns. In this paper, we emphasize the bias on links across different node clusters, which we call cross-links, after considering its significance in both easing information cocoons and preserving graph conne…

Cited by 4SourcePDFScholar
2023

Detecting Backdoors During the Inference Stage Based on Corruption Robustness Consistency

CVPR 2023poster

Deep neural networks are proven to be vulnerable to backdoor attacks. Detecting the trigger samples during the inference stage, i.e., the test-time trigger sample detection, can prevent the backdoor from being triggered. However, existing detection methods often require the defenders to have high ac…

2023

Downstream-agnostic Adversarial Examples

ICCV 2023poster

Self-supervised learning usually uses a large amount of unlabeled data to pre-train an encoder which can be used as a general-purpose feature extractor, such that downstream users only need to perform fine-tuning operations to enjoy the benefit of "big model". Despite this promising prospect, the se…

Cited by 30PDFcodeScholar
2023

ES-Mask: Evolutionary Strip Mask for Explaining Time Series Prediction (Student Abstract)

AAAI 2023technical

Machine learning models are increasingly used in time series prediction with promising results. The model explanation of time series prediction falls behind the model development and makes less sense to users in understanding model decisions. This paper proposes ES-Mask, a post-hoc and model-agnosti…

Cited by 1SourcePDFScholar
2023

PointCA: Evaluating the Robustness of 3D Point Cloud Completion Models against Adversarial Examples

AAAI 2023technical

Point cloud completion, as the upstream procedure of 3D recognition and segmentation, has become an essential part of many tasks such as navigation and scene understanding. While various point cloud completion models have demonstrated their powerful capabilities, their robustness against adversarial…

Cited by 13SourcePDFScholar
2023

SiMFy: A Simple Yet Effective Approach for Temporal Knowledge Graph Reasoning

EMNLP 2023long findings

Temporal Knowledge Graph (TKG) reasoning, which focuses on leveraging temporal information to infer future facts in knowledge graphs, plays a vital role in knowledge graph completion. Typically, existing works for this task design graph neural networks and recurrent neural networks to respectively c…

Cited by 0SourceScholar
2022

Can Language Models Serve as Temporal Knowledge Bases?

EMNLP 2022finding

Recent progress regarding the use of language models (LMs) as knowledge bases (KBs) has shown that language models can act as structured knowledge bases for storing relational facts. However, most existing works only considered the LM-as-KB paradigm in a static setting, which ignores the analysis of…

Cited by 8SourcePDFScholar
2022

OpticE: A Coherence Theory-Based Model for Link Prediction

COLING 2022main

Knowledge representation learning is a key step required for link prediction tasks with knowledge graphs (KGs). During the learning process, the semantics of each entity are embedded by a vector or a point in a feature space. The distance between these points is a measure of semantic similarity. How…

2022

Posistive-Unlabeled Learning via Optimal Transport and Margin Distribution

IJCAI 2022poster

Positive-unlabeled (PU) learning deals with the circumstances where only a portion of positive instances are labeled, while the rest and all negative instances are unlabeled, and due to this confusion, the class prior can not be directly available. Existing PU learning methods usually estimate the c…

Cited by 2SourcePDFScholar
2022

Protecting Facial Privacy: Generating Adversarial Identity Masks via Style-Robust Makeup Transfer

CVPR 2022poster

While deep face recognition (FR) systems have shown amazing performance in identification and verification, they also arouse privacy concerns for their excessive surveillance on users, especially for public face images widely spread on social networks. Recently, some studies adopt adversarial exampl…

Cited by 130PDFcodeScholar
2022

Shielding Federated Learning: Robust Aggregation with Adaptive Client Selection

IJCAI 2022poster

Federated learning (FL) enables multiple clients to collaboratively train an accurate global model while protecting clients' data privacy. However, FL is susceptible to Byzantine attacks from malicious participants. Although the problem has gained significant attention, existing defenses have severa…

2021

Temporal Heterogeneous Information Network Embedding

IJCAI 2021poster

Heterogeneous information network (HIN) embedding, learning the low-dimensional representation of multi-type nodes, has been applied widely and achieved excellent performance. However, most of the previous works focus more on static heterogeneous networks or learning node embedding within specific s…

Cited by 41SourcePDFScholar