An Introduction to Langevin Dynamics: How Stochastic Noise Transforms Optimization and Machine Learning

This short paper introduces Stochastic Gradient Langevin Dynamics (SGLD) as a robust mathematical solution to the limitations of deterministic optimization algorithms, which frequently stagnate in local minima. SGLD utilizes controlled noise, inspired by statistical physics, to enable algorithms to escape suboptimal regions and efficiently explore the global landscape. In this work, we first present the core intuition and mathematical foundations of Stochastic Differential Equations (SDEs) and the Fokker-Planck Equation. Next, we demonstrate how stochasticity overcomes determinism by simulating an annealed trajectory in a double-well potential model. Finally, we highlight two state-of-the-art applications of Langevin Dynamics: escaping high-dimensional saddle points in Deep Learning training, and generating high-fidelity synthetic data via Diffusion Models.