FIACCEL: Memory Efficient Frame Interpolation Accelerator for Full-HD Video

초록

Frame interpolation (FI) is a challenging task that involves generating intermediate frames between two consecutive frames of a video to achieve smooth motion. Although several approaches, including deep learning-based and hybrid methods, have been proposed, most target GPU systems with high computational costs, making it difficult for real-time on-device systems. This brief proposes a memory-efficient and low-complexity accelerator for FI by analyzing the most memory-inefficient part of the encoder-decoder structure and applying schemes such as feature map reuse, selective transfer to DRAM, row-wise layer fusion, kernel decomposition, and parallelized horizontally dilated convolutions. The proposed hardware is verified on an FPGA environment and can synthesize 1920x 1080 video from 90 fps to 180 fps in real-time with an average PSNR quality of 31.98 dB on the Vimeo90K dataset.

키워드