Development Environment of Reinforcement Learning-based Controllers for Real-world Physical Systems Using LW-RCP

초록

In recent years, reinforcement learning (RL)-based controller design methods have emerged as a powerful alternatives to traditional methods, providing a novel paradigm that overcomes limitations associated with the need for accurate model information. In this paper, we propose a development environment for RL-based controllers in real-world systems by integrating MATLAB/Simulink, Python, and the LW-RCP (Light-weight Rapid Control Prototyping) system developed by the authors’ laboratory. The proposed development environment utilizes LW-RCP’s library block in a Simulink-based RL controller model, enabling real-time experiments on real-world systems, and stores state information data in MATLAB’s workspace. Python obtains this data through the Python API after each episode and uses it to iteratively enhance the RL agent’s policy by using RL algorithms. Updated parameter values for the agent’s policy neural network are then sent back to MATLAB's workspace, enabling convenient updates to the deep neural network-based policy controller block in Simulink. This development environment greatly reduces the time and trial and error in configuring real-time system controllers by providing LW-RCP with all necessary functions. Moreover, the efficient data acquisition and integration between MATLAB and Python workspaces facilitate the learning process and reflection of results in Simulink-based controllers. We demonstrate the effectiveness and convenience of the proposed environment through its successful application to the swing-up control problem of a single inverted pendulum. © ICROS 2023.

키워드