Parallel Multi-Dimensional LSTM, With Application to Fast Biomedical Volumetric Image Segmentation
Marijn F Stollenga, Wonmin Byeon, Marcus Liwicki, Jürgen Schmidhuber
Abstract
Convolutional Neural Networks (CNNs) can be shifted across 2D images or 3D videos to segment them. They have a fixed input size and typically perceive only small local contexts of the pixels to be classified as foreground or background. In contrast, Multi-Dimensional Recurrent NNs (MD-RNNs) can perceive the entire spatio-temporal context of each pixel in a few sweeps through all pixels, especially when the RNN is a Long Short-Term Memory (LSTM). Despite these theoretical advantages, however, unlike CNNs, previous MD-LSTM variants were hard to parallelise on GPUs. Here we re-arrange the traditional cuboid order of computations in MD-LSTM in pyramidal fashion. The resulting PyraMiD-LSTM is easy to parallelise, especially for 3D data such as stacks of brain slice images. PyraMiD-LSTM achieved best known pixel-wise brain image segmentation results on MRBrainS13 (and competitive results on EM-ISBI12).
BibTeX
@inproceedings{NIPS2015_d43ab110,
author = {Stollenga, Marijn F and Byeon, Wonmin and Liwicki, Marcus and Schmidhuber, J\"{u}rgen},
booktitle = {Advances in Neural Information Processing Systems},
editor = {C. Cortes and N. Lawrence and D. Lee and M. Sugiyama and R. Garnett},
pages = {},
publisher = {Curran Associates, Inc.},
title = {Parallel Multi-Dimensional LSTM, With Application to Fast Biomedical Volumetric Image Segmentation},
url = {https://proceedings.neurips.cc/paper_files/paper/2015/file/d43ab110ab2489d6b9b2caa394bf920f-Paper.pdf},
volume = {28},
year = {2015}
}