← Search

Jiahe Jin

2 accepted papers

2025

Revisiting 3D LLM Benchmarks: Are We Really Testing 3D Capabilities?

ACL 2025finding

In this work, we identify the “2D-Cheating” problem in 3D LLM evaluation, where these tasks might be easily solved by VLMs with rendered images of point clouds, exposing ineffective evaluation of 3D LLMs’ unique 3D capabilities. We test VLM performance across multiple 3D LLM benchmarks and, using th…