AIConfPaper
← Search

Pieter-Jan Hoedt

4 accepted papers

2026

xLSTM Distillation: Achieving Teacher-Student Parity Through Efficient Hybrid Architectures

Lukas Hauzenberger, Niklas Schmidinger, Thomas Schmied, Anamaria-Roberta Hartl, David Stap, Pieter-Jan Hoedt +3
ICML 2026poster

There have been numerous attempts to distill quadratic attention-based LLMs into sub-quadratic linearized architectures. However, despite extensive research, such distilled models often fail to match the performance of their teacher LLMs on various downstream tasks. We set out the goal of lossless d…

Cited by 0Source ↗Scholar ↗
2025

Bio-xLSTM: Generative modeling, representation and in-context learning of biological and chemical sequences

Niklas Schmidinger, Lisa Schneckenreiter, Philipp Seidl, Johannes Schimunek, Pieter-Jan Hoedt, Johannes Brandstetter +4
ICLR 2025poster

Language models for biological and chemical sequences enable crucial applications such as drug discovery, protein engineering, and precision medicine. Currently, these language models are predominantly based on Transformer architectures. While Transformers have yielded impressive results, their quad…

Cited by 6Source ↗PDF ↗Scholar ↗
2021

MC-LSTM: Mass-Conserving LSTM

Pieter-Jan Hoedt, Frederik Kratzert, Daniel Klotz, Christina Halmich, Markus Holzleitner, Grey S Nearing +2
ICML 2021spotlight

The success of Convolutional Neural Networks (CNNs) in computer vision is mainly driven by their strong inductive bias, which is strong enough to allow CNNs to solve vision-related tasks with random weights, meaning without learning. Similarly, Long Short-Term Memory (LSTM) has a strong inductive bi…

Cited by 107Source ↗PDF ↗code ↗Scholar ↗