Multi-Metric Client Activation Method for Fast and Accurate Federated Learning

초록

While unbiased gradient estimators ensure unbiased solutions in empirical risk minimization problems, they can significantly hinder optimization efficiency and generalization performance in strongly non-IID Federated Learning environments. Although recent studies have demonstrated promising applications of biased estimators, they typically focus only on convergence rates while overlooking generalization performance. We propose a novel multi-metric bias concept, quantified using both local loss and local gradient norm, along with a client activation method based on this bias concept. The proposed method prioritizes training on local datasets that better represent the global dataset, leading to faster convergence and improved generalization. Our extensive empirical study demonstrates that carefully injecting bias into client activation accelerates federated optimization, achieving a substantially improved validation accuracy within a given epoch budget. In representative machine learning benchmarks, our method achieves up to 12.7% higher accuracy than uniform random sampling and 2.5% higher accuracy than state-of-the-art biased client activation methods.

키워드