Reinforcement Learning Applied to Hydrothermal Dispatch

We present an algorithm that combines optimization and reinforcement learning to obtain the optimal dispatch of a multistage hydrothermal system under uncertainty. The model is comprised of neural networks, which provide a target volume for reservoirs at each stage of an optimization subproblem. The optimization problem of each stage minimizes the immediate cost of operating the system and matches the final volume of the reservoirs closely to the target volume. The networks are trained with the DDPG algorithm to minimize the total operating costs for all stages under uncertainty. The model was evaluated in a hydrothermal system with 4 hydroelectric power plants and 95 thermal power plants, comparing the outcomes with the SDDP algorithm results. Because it does not have the same convergence hypotheses as the SDDP, the model allows greater flexibility in modeling the problem. The results show that the proposed algorithm is able to obtain competitive solutions in historical inflow scenarios.