In-Context Learning of Temporal Point Processes with Foundation Inference Models
David Berghaus, Patrick Seifner, Kostadin Cvejoski, Cesar Ojeda, Ramses J Sanchez
Abstract
Modeling event sequences with multiple event types with marked temporal point processes (MTPPs) provides a principled way to uncover governing rules and predict future events. Current neural network approaches to MTPP inference rely on training separate, specialized models for each target system. We pursue a radically different approach: drawing on amortized inference and in-context learning, we pretrain a deep neural network to infer, *in-context*, the conditional intensity functions of event histories from a context defined by sets of event sequences. Pretraining is performed on a large synthetic dataset of MTPPs sampled from a broad distribution over Hawkes processes. Once pretrained, our Foundation Inference Model for Point Processes (FIM-PP) can estimate MTPPs from real-world data without any additional training, or be rapidly finetuned to target systems. Experiments show that this amortized approach matches the performance of specialized models on next-event prediction across common benchmark datasets. We provide the pretrained model weights with the supplementary material.
BibTeX
@inproceedings{
berghaus2026incontext,
title={In-Context Learning of Temporal Point Processes with Foundation Inference Models},
author={David Berghaus and Patrick Seifner and Kostadin Cvejoski and Cesar Ojeda and Ramses J Sanchez},
booktitle={The Fourteenth International Conference on Learning Representations},
year={2026},
url={https://openreview.net/forum?id=h9HwUAODFP}
}