An IREE Compiler-Based SoC Design for Efficient on-Device AI Inference Acceleration

초록

As on-device inference gains traction for privacy protection and low-latency AI services in edge environments, the demand for flexible and lightweight SoC architecture is increasing. This paper proposes a novel SoC design that directly executes IREE (Intermediate Representation Execution Environment) VM Bytecode through an interpreter-based approach implemented on a RISC-V core. The system is validated on a Zynq ZC706 FPGA board and consists of a compiler-hosted PS (Processing System) and an interpreterexecuting PL (Programmable Logic). To improve performance, DMA and cache modules are integrated to optimize data movement and memory access during inference. The proposed system successfully executes various machine learning models, including MUL, MMT, and MNIST, and achieves up to 29 % reduction in inference time depending on the model characteristics. This work demonstrates the feasibility of compiler-driven, reconfigurable inference architectures for embedded AI applications without the need for model-specific hardware redesign. © 2025 IEEE.

키워드