A Double Deep Q-Learning Approach to the Container Relocation Problem

Efficient management of container stacks in maritime terminals is critical to global supply chains, as relocating "blocking'' containers incurs substantial time and cost penalties. This work addresses the static, distinct, deterministic, restricted Container Relocation Problem (CRP), in which containers must be efficiently retrieved in a predetermined order. Following a novel approach, the CRP is modelled as a Markov Decision Process. To learn effective relocation policies without domain-specific heuristics, a Double Deep Q-Learning (DDQN) agent is implemented that leverages neural networks to evaluate the layout of containers and choose the best action. Bayesian Optimization (BO) is implemented to tune crucial hyperparameters systematically and efficiently. Computational experiments on benchmark instances show promising results, with low computational times. This study is, to the best of the authors' knowledge, the first to apply DDQN with BO to the CRP, opening a promising avenue for machine-learning-driven yard management.