Reinforcement learning-based heuristic for static transmission expansion planning in electrical systems

Transmission expansion planning of electrical power systems is, in general, an optimization problem that seeks to identify the set of reinforcements for the electrical transmission network based on desired economic and operational characteristics. Recently, there has been growing interest in applying models that incorporate the set of constraints associated with alternating current power flow, due to the higher quality of the resulting solutions. However, the direct inclusion of these constraints results in a non-convex mixed integer non-linear programming problem, which poses challenges for the application of exact methods. Motivated by recent publications that integrate reinforcement learning into combinatorial optimization problems, this work investigates a heuristic approach based on combining reinforcement learning with a nonlinear solver. Tests carried out on a 6-bus system either reached global optimal objectives or closely approximated results found in the literature.