Janek Haberer

2 accepted papers

2026

ThinkingViT: Matryoshka Thinking Vision Transformer for Elastic Inference

Ali Hojjat, Janek Haberer, Sören Pirk, Olaf Landsiedel

CVPR 2026

ViTs deliver SOTA performance, yet their fixed computational budget prevents scalable deployment across heterogeneous hardware. Recent Matryoshka-style Transformer architectures mitigate this by embedding nested subnetworks within a single model to enable scalable inference. However, these models al

Cited by 0 Source ↗code ↗Scholar ↗

2024

HydraViT: Stacking Heads for a Scalable ViT

Janek Haberer, Ali Hojjat, Olaf Landsiedel

NeurIPS 2024poster

The architecture of Vision Transformers (ViTs), particularly the Multi-head Attention (MHA) mechanism, imposes substantial hardware demands. Deploying ViTs on devices with varying constraints, such as mobile phones, requires multiple models of different sizes. However, this approach has limitations,…

Cited by 0 Source ↗PDF ↗code ↗Scholar ↗