LAB 1: INTRODUCTION TO MULTI-GPU TRAINING
Define a simple neural network and a cost function and iteratively calculate the gradient of the cost function and model parameters using the SGD optimization algorithm.
LAB 2: ALGORITHMIC CHALLENGES TO MULTI GPU TRAINING
Learn to transform single GPU to Horovod multi-GPU implementation to reduce the complexity of writing efficient distributed software. Understand the data loading, augmentation, and training logic using AlexNet model.
LAB 3: ENGINEERING CHALLENGES TO MULTI GPU TRAINING
Understand the aspects of data input pipeline, communication, reference architecture and take a deeper dive into the concepts of job scheduling.