Reward design in multi-agent systems using successor features and multi-information source bayesian optimization

초록

Coordinating self-interested agents in multi-agent systems to achieve system-level objectives presents significant challenges due to the inherent misalignment between individual and collective goals. Mechanism design offers a solution by employing a bi-level optimization framework, where a designer agent intervenes in the reward structures to incentivize desired behaviors among self-interested agents. However, a major obstacle in reward optimization lies in solving multi-agent reinforcement learning problems given a reward structure. This paper addresses this challenge by introducing a novel algorithm that leverages successor features (SFs) at both levels of the optimization. Specifically, SFs help reduce the number of design iterations at the upper level by using previously learned equilibria as biased information sources and accelerate equilibrium learning at the lower level by transferring equilibria from previously solved Markov games. This innovative approach leads to significant computational savings, making the process up to ten times faster compared to traditional methods.

키워드