2023
Fast Saturating Gate for Learning Long Time Scales with Recurrent Neural Networks
AAAI 2023technical
Gate functions in recurrent models, such as an LSTM and GRU, play a central role in learning various time scales in modeling time series data by using a bounded activation function. However, it is difficult to train gates to capture extremely long time scales due to gradient vanishing of the bounded…