SLIDING-MODE CONTROL-BASED OPTIMIZERS FOR DEEP NEURAL NETWORK TRAINING

This work investigates strategies inspired by sliding modes for the training of neural networks. The SMAdam and SMSGD optimizers are proposed and three experiments are carried out: comparison between classical backpropagation and SMC-BP, classification in FashionMNIST and prediction of time series with the ETT dataset. The results suggest that sliding surfaces are promising as a compromise mechanism between convergence speed, robustness and training cost.