Jianwei Feng

Optimal Gradient Checkpoint Search for Arbitrary Computation Graphs

Deep Neural Networks(DNNs) require huge GPU memory when training on modern image/video databases. Unfortunately, the GPU memory is physically finite, which limits the image resolutions and batch sizes that could be used in training for better DNN performance. Unlike solutions that require physically…