Full-Reference Point Cloud Quality Assessment with Multimodal Large Language Models

Abstract

Point cloud quality frequently degrades during various processes, such as scanning, compression, and transmission. Hence, reliable Point Cloud Quality Assessment (PCQA) methods are essential for detecting and mitigating the degradation in 3D applications. This paper proposes an accurate full-reference PCQA method that leverages Multimodal Large Language Models (MLLMs). The proposed method utilizes responses generated by MLLMs to assess point cloud quality. We introduce three innovative PCQA metrics derived from MLLMs: 1) response similarity score, 2) relative quality response score, and 3) absolute quality response score. In addition, we integrate these MLLM-based scores with conventional PCQA metrics using support vector regression to improve accuracy. Experimental results demonstrate that the average Pearson’s Linear Correlation Coefficient (PLCC) and Spearman’s Rank-Order Correlation Coefficient (SROCC) across three datasets improved by 0.046 (from 0.871 to 0.917) and 0.055 (from 0.842 to 0.897), respectively, compared to the state-of-the-art FR-PCQA method.