2024
Harnessing Manycore Processors with Distributed Memory for Accelerated Training of Sparse and Recurrent Models
AAAI 2024technical
Current AI training infrastructure is dominated by single instruction multiple data (SIMD) and systolic array architectures, such as Graphics Processing Units (GPUs) and Tensor Processing Units (TPUs), that excel at accelerating parallel workloads and dense vector matrix multiplications. Potentially…