← Search

Qinkai Zheng

4 accepted papers

2025

VisualAgentBench: Towards Large Multimodal Models as Visual Foundation Agents

ICLR 2025poster

Large Multimodal Models (LMMs) have ushered in a new era in artificial intelligence, merging capabilities in both language and vision to form highly capable \textbf{Visual Foundation Agents} that are postulated to excel across a myriad of tasks. However, existing benchmarks fail to sufficiently chal…

2024

NaturalCodeBench: Examining Coding Performance Mismatch on HumanEval and Natural User Queries

ACL 2024findings

Large language models (LLMs) have manifested strong ability to generate codes for productive activities. However, current benchmarks for code synthesis, such as HumanEval, MBPP, and DS-1000, are predominantly oriented towards introductory tasks on algorithm and data science, insufficiently satisfyin…

2024

OctoPack: Instruction Tuning Code Large Language Models

ICLR 2024spotlight

Finetuning large language models (LLMs) on instructions leads to vast performance improvements on natural language tasks. We apply instruction tuning using code, leveraging the natural structure of Git commits, which pair code changes with human instructions. We compile CommitPack: 4 terabytes of Gi…

2021

Graph Robustness Benchmark: Benchmarking the Adversarial Robustness of Graph Machine Learning

NeurIPS 2021poster

Adversarial attacks on graphs have posed a major threat to the robustness of graph machine learning (GML) models. Naturally, there is an ever-escalating arms race between attackers and defenders. However, the strategies behind both sides are often not fairly compared under the same and realistic con…

Cited by 57SourcecodeScholar