CPsyExam: A Chinese Benchmark for Evaluating Psychology using Examinations
Jiahao Zhao, Jingwei Zhu, Minghuan Tan, Min Yang, Renhao Li, Yang Di, Chenhao Zhang, Guancheng Ye
Abstract
In this paper, we introduce a novel psychological benchmark, CPsyExam, constructed from questions sourced from Chinese examination systems. CPsyExam is designed to prioritize psychological knowledge and case analysis separately, recognizing the significance of applying psychological knowledge to real-world scenarios. We collect 22k questions from 39 psychology-related subjects across four Chinese examination systems. From the pool of 22k questions, we utilize 4k to create the benchmark that offers balanced coverage of subjects and incorporates a diverse range of case analysis techniques. Furthermore, we evaluate a range of existing large language models (LLMs), spanning from open-sourced to proprietary models. Our experiments and analysis demonstrate that CPsyExam serves as an effective benchmark for enhancing the understanding of psychology within LLMs and enables the comparison of LLMs across various granularities.
BibTeX
@inproceedings{zhao-etal-2025-cpsyexam,
title = "{CP}sy{E}xam: A {C}hinese Benchmark for Evaluating Psychology using Examinations",
author = "Zhao, Jiahao and
Zhu, Jingwei and
Tan, Minghuan and
Yang, Min and
Li, Renhao and
Di, Yang and
Zhang, Chenhao and
Ye, Guancheng and
Li, Chengming and
Hu, Xiping and
Wong, Derek F.",
editor = "Rambow, Owen and
Wanner, Leo and
Apidianaki, Marianna and
Al-Khalifa, Hend and
Eugenio, Barbara Di and
Schockaert, Steven",
booktitle = "Proceedings of the 31st International Conference on Computational Linguistics",
month = jan,
year = "2025",
address = "Abu Dhabi, UAE",
publisher = "Association for Computational Linguistics",
url = "https://aclanthology.org/2025.coling-main.745/",
pages = "11248--11260"
}