Optimizing Large Language Model Responses via a Biased Random-Key Genetic Algorithm

This paper addresses the optimization of responses generated by Large Language Models (LLMs) at inference time. While current state-of-the-art approaches often rely on optimal stopping frameworks or combinatorial methods that operate over the prompt space, these techniques are frequently constrained by the high cost of multiple LLM calls. We propose a novel heuristic framework based on a Biased Random-Key Genetic Algorithm (BRKGA) capable of performing active search within the response vector space. Our method optimizes candidate responses at the lexical level, guided by a semantic reward model, to identify high-quality neighborhoods of an initial generation. Experimental results demonstrate that the proposed pipeline outperforms the Best-of-5 sampling baseline in 70% of the evaluated instances. Furthermore, our approach achieves a 60% reduction in computational costs, requiring significantly fewer LLM inference calls while maintaining superior response quality and grammatical coherence.