Deep Reinforcement Learning-Based Well Placement Optimization Using a Time Series Proxy Model

초록

This study introduces a framework for optimizing well placement using deep reinforcement learning (RL) and a time-series proxy model. The proxy model predicts net present value over time and leverages time-of-flight maps to capture fluid drainage areas, enhancing robustness against geological uncertainties. Tested on randomly generated scenarios, the proxy model achieved an average coefficient of determination (R2) of 0.98 for production-only cases and 0.92 when injection wells were included, demonstrating its predictive accuracy under varying conditions. Using the proxy model's predictions, the RL framework identified well placements with similar or higher economic value compared to particle swarm optimization. Additionally, the framework required only 47% of the computational cost compared to using RL alone. Enhancements such as double Q-learning, prioritized experience replay, and action masking improved the framework's efficiency and decision-making accuracy. These results demonstrate the framework's ability to optimize field development planning while addressing computational challenges. By integrating artificial intelligence techniques with practical reservoir engineering applications, this study provides a scalable approach for well placement optimization, offering a balance between efficiency and solution quality. © 2025 86th EAGE Annual Conference and Exhibition. All rights reserved.