AISTATS 2021poster28 citations
Deep Fourier Kernel for Self-Attentive Point Processes
Shixiang Zhu, Minghe Zhang, Ruyi Ding, Yao Xie
Abstract
We present a novel attention-based model for discrete event data to capture complex non-linear temporal dependence structures. We borrow the idea from the attention mechanism and incorporate it into the point processes’ conditional intensity function. We further introduce a novel score function using Fourier kernel embedding, whose spectrum is represented using neural networks, which drastically differs from the traditional dot-product kernel and can capture a more complex similarity structure. We establish our approach’s theoretical properties and demonstrate our approach’s competitive performance compared to the state-of-the-art for synthetic and real data.
BibTeX
@InProceedings{pmlr-v130-zhu21b,
title = { Deep Fourier Kernel for Self-Attentive Point Processes },
author = {Zhu, Shixiang and Zhang, Minghe and Ding, Ruyi and Xie, Yao},
booktitle = {Proceedings of The 24th International Conference on Artificial Intelligence and Statistics},
pages = {856--864},
year = {2021},
editor = {Banerjee, Arindam and Fukumizu, Kenji},
volume = {130},
series = {Proceedings of Machine Learning Research},
month = {13--15 Apr},
publisher = {PMLR},
pdf = {http://proceedings.mlr.press/v130/zhu21b/zhu21b.pdf},
url = {https://proceedings.mlr.press/v130/zhu21b.html},
abstract = { We present a novel attention-based model for discrete event data to capture complex non-linear temporal dependence structures. We borrow the idea from the attention mechanism and incorporate it into the point processes’ conditional intensity function. We further introduce a novel score function using Fourier kernel embedding, whose spectrum is represented using neural networks, which drastically differs from the traditional dot-product kernel and can capture a more complex similarity structure. We establish our approach’s theoretical properties and demonstrate our approach’s competitive performance compared to the state-of-the-art for synthetic and real data. }
}