Stable Quantization-Aware Training with Adaptive Gradient Clipping

초록

Quantization-Aware Training (QAT) uses batch normalization (BN) folding during fine-tuning, so it may not use the normalization effects of the BN layer. HAWQ, which achieved SOTA with QAT, has significant accuracy degradation as the training becomes longer. In this paper, we apply Adaptive Gradient Clipping (AGC) to stable quantization-aware training and improve accuracy by adding Dropout. Moreover, we have ablation studies about AGC. © 2023 IEEE.