Last-Mile Delivery under Uncertainty: Simulation and Solution with Reinforcement Learning and Heuristics

Last-mile delivery in scenarios of stochastic demand imposes challenges on urban logistics, particularly in the efficient real-time allocation of drivers. This work investigates the application of the Proximal Policy Optimization (PPO) algorithm and heuristics to the dynamic driver allocation problem, formalized as a Markov Decision Process. Contributions include the formalization of the problem, a modular discrete-event simulator, and a systematic comparison between heuristics and PPO agents in scenarios of increasing complexity. PPO with hyperparameter optimization outperforms heuristics in higher-demand scenarios, with reductions of 20% and 55% in effective delivery time.