Jun Niu

2 accepted papers

2025

Spiking Transformer with Spatial-Temporal Spiking Self-Attention

Zhaokun Zhou, Jun Niu, Yang Zhang, Li Yuan, Yuesheng Zhu

ICASSP 2025accepted

Spiking Neural Networks are celebrated for energy efficiency and biological plausibility. Building on Spiking Self-Attention (SSA), Spiking Transformers are extensively studied due to their exceptional performance. However, SSA focuses solely on spatial dimension at each time step, overlooking the c…

Cited by 0 Source ↗Scholar ↗

2024

Spiking Transformer with Experts Mixture

Zhaokun Zhou, Yijie Lu, Yanhao Jia, Kaiwei Che, Jun Niu, Liwei Huang +5

NeurIPS 2024poster

Spiking Neural Networks (SNNs) provide a sparse spike-driven mechanism which is believed to be critical for energy-efficient deep learning. Mixture-of-Experts (MoE), on the other side, aligns with the brain mechanism of distributed and sparse processing, resulting in an efficient way of enhancing m…

Cited by 1 Source ↗PDF ↗Scholar ↗