Substate-based UAV Movement Control for Data Collection in WSN Using Q-Learning

초록

In this paper, we propose an unmanned aerial vehicle (UAV) movement control for data gathering in a wireless sensor network (WSN) using a Q-learning algorithm. The value of the acquired sensor data depends on time. Since the sensing value recovery time is a critical component in data collecting, we propose a substate-based UAV path control approach to maximize the value of the acquired sensing data from the ground sensors. The value of the collected data depends on the number of sensors in the UAV coverage and the elapsed time after acquiring previous sensing data from the current coverage. We use the Q-learning algorithm to maximize the total reward. To capture the effect of the elapsed time we define a substate concept. Each state stores multiple substates where only one of them is active at a given time. Consequently, a Q-value in the Q-table is updated for each substate separately. The main goal of this paper is to maximize the sensing information within the given initial energy during real-time UAV operation. Experiments show that the proposed method increased the rewards of UAV operation compared with the conventional approach. © 2022 IEEE.

키워드