Analysis of the Impact of Parallel Data Loading on Convolutional Neural Network Training

Training convolutional neural networks for medical image classification is limited not
only by the computing capacity of the graphics processor, but also by the speed at which data are
read and prepared in memory. This work investigates to what extent parallel data loading reduces
this bottleneck, and at which point it ceases to pay off. Models were trained on the HAM10000
collection of dermatoscopic images, varying the number of loading processes between 0 and 8 and
the batch size between 32 and 256, across two distinct environments: a personal computer and the
free Google Colab platform. On the former, parallelism reduced the time per epoch from 21.66 s
to 4.09 s, a 5.3-fold gain, saturating once the number of processes matches the number of available
physical cores; on the latter, the maximum gain was only 1.23-fold, with degradation above two
processes. Batch size had a negligible effect on execution time. We conclude that the number of
processes must be calibrated empirically for each environment, this being a low-cost, high-return
measure for accelerating training.