Learning Search Behavior in Random-Key Space with Deep Reinforcement Learning for Binary Combinatorial Optimization

Combinatorial optimization problems have increasingly been addressed using Deep Reinforcement Learning (DRL) due to its ability to learn decision policies directly from experience. However, most DRL models applied to optimization are structured as construction heuristics, where agents build solutions sequentially and often become specialized to the distribution of a specific problem. In contrast, traditional metaheuristics are typically problem-independent but rely on stochastic search mechanisms and predefined hyperparameters without adapting their search behavior from previous exploration. To address this gap, this paper proposes a DRL-based architecture structured upon the continuous hypercube of Random-Key Optimizers (RKOs). By separating the continuous search process from the deterministic decoding procedure, the model allows the agent to perform iterative search in the continuous space while remaining agnostic to the underlying binary optimization problem. Experimental results show that the learned search policy transfers in a zero-shot setting across unseen binary NP-hard problems without additional training.