CLAM-Bench: Benchmarking LLM Agents for Library-Scale Cross-Architecture Migration
Weijia Li, KE GAO, Jiajie Li, Han Sun, Yuhe Ding, Yongdong Mai, Yiran Le, Yongjie Qian
Abstract
Cross-architecture migration of high-performance libraries dictates ecosystem readiness on emerging hardware. The challenge is twofold: disentangling library-scale dependencies and performance-critical kernels with ISA-specific SIMD intrinsics, often trading migration speed for peak performance. While LLM-based agents offer a promising approach, are confined to function-level tasks or scalar code, failing to assess agents’ capabilities and limitations in realistic, library-scale migration. We present CLAM-Bench (Cross-architecture Library-scale Agent Migration benchmark), featuring 85 critical kernels from widely used libraries, including OpenCV, libjpeg, and NCNN. It supports comprehensive evaluations of compilability, correctness, and performance across major transitions: ARM→RISC-V, x86→ARM, and ARM→LoongArch. Evaluation of 12 SOTA agent-LLM combinations on CLAM-Bench reveals that, due to the lack of library-level navigation and hardware-aware optimization, agents regress to superficial pattern matching, yielding only 20.88% correctness and 0.83x speedup for libjpeg. Motivated by these findings, we further propose FSCM, a multi-agent framework incorporating hardware-aware global reconfiguration and performance optimization. FSCM improves OpenCV correctness to 71%. The benchmark and code are available at https://anonymous.4open.science/r/clam_bench-D8EB/.
BibTeX
@inproceedings{
li2026qimenglibbench,
title={QiMeng-LibBench: Benchmarking {LLM} Agents for Library-Scale Cross-Architecture Migration},
author={Weijia Li and KE GAO and Jiajie Li and Han Sun and Yuhe Ding and Yongdong Mai and Yiran Le and Yongjie Qian and Zhibin Zhang and Xinyu Wang and Limin Cheng and Shouxu Kuang and Pengfei Chen and Ling Li},
booktitle={Forty-third International Conference on Machine Learning},
year={2026},
url={https://openreview.net/forum?id=uhSqCIVc0B}
}