VecIntrinBench: Benchmarking Cross-Architecture Intrinsic Code Migration for RISC-V Vector

TL;DR

VecIntrinBench introduces the first RVV-inclusive benchmark for cross-architecture intrinsic code migration and uses 50 real-world use cases with scalar, RVV, Arm Neon, and x86 intrinsics. It provides correctness and performance test suites, with evaluation metrics based on $pass@k$ and $speedup$, enabling a direct comparison of rule-based and LLM-based migration. Results show that advanced LLMs achieve higher correctness and more favorable $speedup$ distributions than the rule-based neon2rvv baseline, yet still face challenges from RVV semantic gaps and hardware-specific nuances. The open-source benchmark enables systematic study and guides future work on hybrid migration strategies and domain-specific model fine-tuning to advance cross-architecture vectorization.

Abstract

Intrinsic functions are specialized functions provided by the compiler that efficiently operate on architecture-specific hardware, allowing programmers to write optimized code in a high-level language that fully exploits hardware features. Using intrinsics to vectorize core code blocks is a standard optimization method in high-performance libraries, often requiring specific vector optimization implementations for multiple mainstream architectures. The promising RISC-V software ecosystem has a significant demand for algorithm library migration and adaptation. Translating existing intrinsic functions to RISC-V Vector (RVV) intrinsic functions across architectures is currently a mainstream approach. Rule-based intrinsic mapping methods and LLM-based code generation can help developers address the code migration challenge. However, existing intrinsic code benchmarks focus on mainstream SIMD intrinsics and lack support for the emerging RISC-V architecture. There is currently no benchmark that comprehensively evaluates the intrinsic migration capabilities for the RVV extension. To fill this gap, we propose VecIntrinBench, the first intrinsic benchmark encompassing RVV extensions. It includes 50 function-level tasks from open source repositories, implemented as scalars, RVV intrinsics, Arm Neon intrinsics, and x86 intrinsics, along with comprehensive functional and performance test cases. We systematically evaluated various code migration approaches on VecIntrinBench, yielding a series of insightful findings. The results demonstrate that advanced Large Language Models (LLMs) achieve a similar effect as rule-based mapping approaches for RISC-V code migration, while also delivering superior performance. We further analyze the reasons and identify future directions for LLM development in the code migration field. The VecIntrinBench is open-sourced to benefit the broader community and developers.

Figures (4)

• Figure 1: Example of Migrating Arm Neon Intrinsic Code to RVV
• Figure 2: Overview of a Case in VecIntrinBench
• Figure 3: Effectiveness of code migration methods at different levels of k.
• Figure 4: Performance Speedup Distribution for Passed Vector Intrinsics.