Mask-based adaptive response distillation for efficient image super-resolution

초록

Knowledge distillation (KD) is a promising strategy for lightweight image super-resolution (ISR). However, most existing methods rely on vanilla L 1 response distillation, which fails to account for the importance of high-frequency structures and treats all pixels equally-often leading to suboptimal learning. To address these issues, we propose a simple yet effective KD framework that is both cost-efficient and architecture-agnostic. Our method introduces: (1) a mask-based adaptive response distillation strategy that emphasizes hard instances and pixels via instance-wise suppression and pixel-wise weighting; and (2) a GT-pretrained student initialization scheme that improves optimization stability and performance. Unlike prior works that add significant training overhead, our framework enhances KD effectiveness without requiring additional modules or labels. Extensive experiments on both CNN and Transformer-based SR models demonstrate that our method consistently outperforms baseline and state-of-the-art KD techniques across multiple datasets and upscaling factors, offering strong generalization and plug-in flexibility.

키워드