← Search

Shijin Wang

44 accepted papers

2026

CARE-Bench: A Benchmark of Diverse Client Simulations Guided by Expert Principles for Evaluating LLMs in Psychological Counseling

AAAI 2026technical

The mismatch between the growing demand for psychological counseling and the limited availability of services has motivated research into the application of Large Language Models (LLMs) in this domain. Consequently, there is a need for a robust and unified benchmark to assess the counseling competen

Cited by 0SourcePDFScholar
2026

ChemEval: A Multi-level and Fine-grained Chemical Capability Evaluation for Large Language Models

ICLR 2026poster

The emergence of Large Language Models (LLMs) in chemistry marks a significant advancement in applying artificial intelligence to chemical sciences. While these models show promising potential, their effective application in chemistry demands sophisticated evaluation protocols that address the field…

Cited by 0SourcecodeScholar
2026

Fewer Battles, More Gain: An Information-Efficient Framework for Arena-based LLM Evaluation

ICLR 2026poster

Arena-based evaluation has become a key method for assessing large language models (LLMs) through head-to-head model comparisons, closely reflecting human preferences. However, current arena rating systems (e.g., ELO rating system) often suffer from inefficiencies due to exhaustive or random model p…

Cited by 0SourcecodeScholar
2026

From Diagnosis to Generalization: A Cognitive Approach to Data Selection for Educational LLMs

AAAI 2026technical

Specializing Large Language Models for educational domains is a key frontier in creating personalized learning tools. The central challenge is not data scarcity but its abundance: efficiently selecting a curated data subset from vast corpora to enhance specialized skills and foster generalization, w

Cited by 0SourcePDFScholar
2026

From Values to Tokens: An LLM-Driven Framework for Context-Aware Time Series Forecasting via Symbolic Discretization

IJCAI 2026

Time series forecasting plays a vital role in supporting decision-making across a wide range of critical applications, including energy, healthcare, and finance. Despite recent advances, forecasting accuracy remains limited due to the challenge of integrating historical numerical sequences with cont

Cited by 0Scholar
2026

MemCast: Memory-Driven Time Series Forecasting with Experience-Conditioned Reasoning

ICML 2026poster

Time series forecasting (TSF) plays a critical role in decision-making for many real-world applications. Recently, LLM-based forecasters have made promising advancements. Despite their effectiveness, existing methods often lack explicit experience accumulation and continual evolution. In this work, …

Cited by 0SourceScholar
2025

Chain of Strategy Optimization Makes Large Language Models Better Emotional Supporter

EMNLP 2025

The growing emotional stress in modern society has increased the demand for Emotional Support Conversations (ESC). While Large Language Models (LLMs) show promise for ESC, they face two key challenges: (1) low strategy selection accuracy, and (2) preference bias, limiting their adaptability to users

Cited by 0SourcePDFScholar
2025

Chart2Code53: A Large-Scale Diverse and Complex Dataset for Enhancing Chart-to-Code Generation

EMNLP 2025

Chart2code has recently received significant attention in the multimodal community due to its potential to reduce the burden of visualization and promote a more detailed understanding of charts. However, existing Chart2code-related training datasets suffer from at least one of the following issues:

2025

CogMath: Assessing LLMs' Authentic Mathematical Ability from a Human Cognitive Perspective

ICML 2025poster

Although large language models (LLMs) show promise in solving complex mathematical tasks, existing evaluation paradigms rely solely on a coarse measure of overall answer accuracy, which are insufficient for assessing their authentic capabilities. In this paper, we propose \textbf{CogMath}, which com…

Cited by 0SourcePDFScholar
2025

CursorCore: Assist Programming through Aligning Anything

ICML 2025poster

Large language models have been successfully applied to programming assistance tasks, such as code completion, code insertion, and instructional code editing. However, these applications remain insufficiently automated and struggle to effectively integrate various types of information during the pro…

2025

Enhancing Chain-of-Thought Reasoning via Neuron Activation Differential Analysis

EMNLP 2025

Despite the impressive chain-of-thought(CoT) reasoning ability of large language models (LLMs), its underlying mechanisms remains unclear. In this paper, we explore the inner workings of LLM’s CoT ability via the lens of neurons in the feed-forward layers. We propose an efficient method to identify

Cited by 0SourcePDFScholar
2025

Evaluating LLMs Across Multi-Cognitive Levels: From Medical Knowledge Mastery to Scenario-Based Problem Solving

ICML 2025poster

Large language models (LLMs) have demonstrated remarkable performance on various medical benchmarks, but their capabilities across different cognitive levels remain underexplored. Inspired by Bloom's Taxonomy, we propose a multi-cognitive-level evaluation framework for assessing LLMs in the medical…

2025

Evaluating Large Language Models through Role-Guide and Self-Reflection: A Comparative Study

ICLR 2025poster

Large Language Models fine-tuned with Reinforcement Learning from Human Feedback (RLHF-LLMs) can over-rely on aligned preferences without truly gaining self-knowledge, leading to hallucination and biases. If an LLM can better access its knowledge and know what it knows, it can avoid making false or…

Cited by 2SourcePDFScholar
2025

FACT: Mitigating Inconsistent Hallucinations in LLMs via Fact-Driven Alternating Code-Text Training

NeurIPS 2025poster

Inconsistent hallucinations remain a major challenge for large language models (LLMs), undermining the accuracy and reliability of fact-based reasoning in real-world applications. Existing approaches often rely on task-specific training or adaptation, such as hand-crafted synthetic datasets for doma…

Cited by 0SourceScholar
2025

From Objectives to Questions: A Planning-based Framework for Educational Mathematical Question Generation

ACL 2025long

Automatically generating high-quality mathematical problems that align with educational objectives is a crucial task in NLP-based educational technology. Traditional generation methods focus primarily on textual quality, but they often overlook educational objectives. Moreover, these methods address…

Cited by 0SourcePDFScholar
2025

How Does Sequence Modeling Architecture Influence Base Capabilities of Pre-trained Language Models? Exploring Key Architecture Design Principles to Avoid Base Capabilities Degradation

NeurIPS 2025poster

Pre-trained language models represented by the Transformer have been proven to possess strong base capabilities, and the representative self-attention mechanism in the Transformer has become a classic in sequence modeling architectures. Different from the work of proposing sequence modeling architec…

Cited by 0SourceScholar
2025

Investigating and Mitigating Catastrophic Forgetting in Medical Knowledge Injection through Internal Knowledge Augmentation Learning

NeurIPS 2025poster

Large Language Models (LLMs) are expected to possess comprehensive medical knowledge to support real-world clinical applications. While domain-specific fine-tuning effectively injects medical knowledge into LLMs, it often causes catastrophic forgetting of previously acquired knowledge and instructio…

Cited by 0SourcecodeScholar
2025

Multi-Perspective Consolidation Enhanced Cognitive Diagnosis via Conditional Diffusion Model

AAAI 2025technical

Cognitive diagnosis, which assesses the learners' competence from learners' interaction logs, plays a vital role in education. It provides a crucial reference for gauging learners' proficiency levels and tailoring future learning activities accordingly. Researchers have proposed numerous cognitive d…

2025

Ontology-Guided Reverse Thinking Makes Large Language Models Stronger on Knowledge Graph Question Answering

ACL 2025long

Large language models (LLMs) have shown remarkable capabilities in natural language processing. However, in knowledge graph question answering tasks (KGQA), there remains the issue of answering questions that require multi-hop reasoning. Existing methods rely on entity vector matching, but the purpo…

Cited by 0SourcePDFScholar
2025

Position: AI Evaluation Should Learn from How We Test Humans

ICML 2025poster

As AI systems continue to evolve, their rigorous evaluation becomes crucial for their development and deployment. Researchers have constructed various large-scale benchmarks to determine their capabilities, typically against a gold-standard test set and report metrics averaged across all items. Howe…

2025

am-ELO: A Stable Framework for Arena-based LLM Evaluation

ICML 2025spotlight

Arena-based evaluation is a fundamental yet significant evaluation paradigm for modern AI models, especially large language models (LLMs). Existing framework based on ELO rating system suffers from the inevitable instability problem due to ranking inconsistency and the lack of attention to the varyi…

Cited by 0SourcePDFScholar
2024

CONSIDER: Commonalities and Specialties Driven Multilingual Code Retrieval Framework

AAAI 2024technical

Multilingual code retrieval aims to find code snippets relevant to a user's query from a multilingual codebase, which plays a crucial role in software development and expands their application scenarios compared to classical monolingual code retrieval. Despite the performance improvements achieved b…

2024

Computerized Adaptive Testing via Collaborative Ranking

NeurIPS 2024poster

As the deep integration of machine learning and intelligent education, Computerized Adaptive Testing (CAT) has received more and more research attention. Compared to traditional paper-and-pencil tests, CAT can deliver both personalized and interactive assessments by automatically adjusting testing q…

Cited by 1SourcePDFScholar
2024

JiuZhang3.0: Efficiently Improving Mathematical Reasoning by Training Small Data Synthesis Models

NeurIPS 2024poster

Mathematical reasoning is an important capability of large language models~(LLMs) for real-world applications. To enhance this capability, existing work either collects large-scale math-related texts for pre-training, or relies on stronger LLMs (\eg GPT-4) to synthesize massive math problems. Both t…

2024

Learning to Solve Geometry Problems via Simulating Human Dual-Reasoning Process

IJCAI 2024poster

Geometry Problem Solving (GPS), which is a classic and challenging math problem, has attracted much attention in recent years. It requires a solver to comprehensively understand both text and diagram, master essential geometry knowledge, and appropriately apply it in reasoning. However, existing wor…

2024

Optimizing Chinese Lexical Simplification Across Word Types: A Hybrid Approach

EMNLP 2024main

This paper addresses the task of Chinese Lexical Simplification (CLS). A key challenge in CLS is the scarcity of data resources. We begin by evaluating the performance of various language models at different scales in unsupervised and few-shot settings, finding that their effectiveness is sensitive…

2024

SocraticLM: Exploring Socratic Personalized Teaching with Large Language Models

NeurIPS 2024spotlight

Large language models (LLMs) are considered a crucial technology for advancing intelligent education since they exhibit the potential for an in-depth understanding of teaching scenarios and providing students with personalized guidance. Nonetheless, current LLM-based application in personalized teac…

2024

SparkRA: A Retrieval-Augmented Knowledge Service System Based on Spark Large Language Model

EMNLP 2024system demonstrations

Large language models (LLMs) have shown remarkable achievements across various language tasks. To enhance the performance of LLMs in scientific literature services, we developed the scientific literature LLM (SciLit-LLM) through pre-training and supervised fine-tuning on scientific literature, build…

Cited by 1SourcePDFScholar
2024

Towards Accurate and Fair Cognitive Diagnosis via Monotonic Data Augmentation

NeurIPS 2024poster

Intelligent education stands as a prominent application of machine learning. Within this domain, cognitive diagnosis (CD) is a key research focus that aims to diagnose students' proficiency levels in specific knowledge concepts. As a crucial task within the field of education, cognitive diagnosis en…

Cited by 2SourcePDFScholar
2023

A Facial Expression-Aware Multimodal Multi-task Learning Framework for Emotion Recognition in Multi-party Conversations

ACL 2023long

Multimodal Emotion Recognition in Multiparty Conversations (MERMC) has recently attracted considerable attention. Due to the complexity of visual scenes in multi-party conversations, most previous MERMC studies mainly focus on text and audio modalities while ignoring visual information. Recently, se…

2023

BETA-CD: A Bayesian Meta-Learned Cognitive Diagnosis Framework for Personalized Learning

AAAI 2023technical

Personalized learning is a promising educational approach that aims to provide high-quality personalized services for each student with minimum demands for practice data. The key to achieving that lies in the cognitive diagnosis task, which estimates the cognitive state of the student through his/he…

2023

Chinese Metaphorical Relation Extraction

EMNLP 2023long findings

Metaphors are linguistic expressions that convey non-literal meanings, as well as cognitive mappings that establish connections between distinct domains of experience or knowledge. This paper proposes a novel formulation of metaphor identification as a relation extraction problem. We introduce meta…

Cited by 0SourcecodeScholar
2023

Evaluating and Improving Tool-Augmented Computation-Intensive Math Reasoning

NeurIPS 2023poster

Chain-of-thought prompting (CoT) and tool augmentation have been validated in recent work as effective practices for improving large language models (LLMs) to perform step-by-step reasoning on complex math-related tasks. However, most existing math reasoning datasets may not be able to fully evaluat…

2023

Exploiting Non-Interactive Exercises in Cognitive Diagnosis

IJCAI 2023poster

Cognitive Diagnosis aims to quantify the proficiency level of students on specific knowledge concepts. Existing studies merely leverage observed historical students-exercise interaction logs to access proficiency levels. Despite effectiveness, observed interactions usually exhibit a power-law distri…

2023

Gradient-based Intra-attention Pruning on Pre-trained Language Models

ACL 2023long

Pre-trained language models achieve superior performance but are computationally expensive. Techniques such as pruning and knowledge distillation have been developed to reduce their sizes and latencies. In this work, we propose a structured pruning method GRAIN (gradient-based intra-attention prunin…

2023

IDOL: Indicator-oriented Logic Pre-training for Logical Reasoning

ACL 2023findings

In the field of machine reading comprehension (MRC), existing systems have surpassed the average performance of human beings in many tasks like SQuAD. However, there is still a long way to go when it comes to logical reasoning. Although some methods for it have been put forward, they either are desi…

2023

Towards a Holistic Understanding of Mathematical Questions with Contrastive Pre-training

AAAI 2023technical

Understanding mathematical questions effectively is a crucial task, which can benefit many applications, such as difficulty estimation. Researchers have drawn much attention to designing pre-training models for question representations due to the scarcity of human annotations (e.g., labeling difficu…

2022

Continual Pre-training of Language Models for Math Problem Understanding with Syntax-Aware Memory Network

ACL 2022long

In this paper, we study how to continually pre-train language models for improving the understanding of math problems. Specifically, we focus on solving a fundamental challenge in modeling math problems, how to fuse the semantics of textual description and formulas, which are highly different in ess…

2021

HMS: A Hierarchical Solver with Dependency-Enhanced Understanding for Math Word Problem

AAAI 2021technical

Automatically solving math word problems is a crucial task for exploring the intelligence levels of machines in the general AI domain. It is highly challenging since it requires not only natural language understanding but also mathematical expression inference. Existing solutions usually explore seq…

2020

A Sentence Cloze Dataset for Chinese Machine Reading Comprehension

COLING 2020main

Owing to the continuous efforts by the Chinese NLP community, more and more Chinese machine reading comprehension datasets become available. To add diversity in this area, in this paper, we propose a new task called Sentence Cloze-style Machine Reading Comprehension (SC-MRC). The proposed task aims…

2020

CharBERT: Character-aware Pre-trained Language Model

COLING 2020main

Most pre-trained language models (PLMs) construct word representations at subword level with Byte-Pair Encoding (BPE) or its variations, by which OOV (out-of-vocab) words are almost avoidable. However, those methods split a word into subword units and make the representation incomplete and fragile.…

2020

Text Classification by Contrastive Learning and Cross-lingual Data Augmentation for Alzheimer’s Disease Detection

COLING 2020main

Data scarcity is always a constraint on analyzing speech transcriptions for automatic Alzheimer’s disease (AD) detection, especially when the subjects are non-English speakers. To deal with this issue, this paper first proposes a contrastive learning method to obtain effective representations for te…