Creating a Reinforcement Learning Environment with an Ultra-Low-Cost Handmade Self-Driving Simulator and its Application to Deep-Learning

초록

In this paper, we use a low cost driving simulator to create a reinforcement learning environment for the training of a deep learning algorithm of self-driving cars. The states, action and reward according to the Markov Decision Process (MDP) are defined and implemented in the simulator. In the case of the states, the speed and heading, acceleration, brake, position, the state of the vehicle, and the images on the current screen are included. For actions, continuous values such as steering angle, acceleration, and brake are used for a driving vehicle. However, for simplicity in this study, discrete input values such as go, stop, turn left and turn right provided by a human operator are employed. In the case of rewards, if the vehicle goes well along the planned path, the bonus is given. On the other hand, if the vehicle invades the lane or hits surrounding objects, the penalty is given. At this time, waypoints are an important tool to calculate rewards. In this study, a certain region in the simulation scenario was selected as a test region, and the waypoints in the region were acquired for the path of interest. After virtual simulation environment creation, we use a Deep Q Network (DQN) algorithm for reinforcement learning by using images on screen for end-to-end self-driving algorithms. To verify the performance of