Juhyeong Kim

FractalLLM: Lossless Self-Speculative Decoding with Layer Embedded Self-Compression

Autoregressive decoding in large language models (LLMs) necessitates a full forward pass for each generated token, significantly increasing inference latency. To address this limitation, we propose Fractal-LLM, a lossless self-speculative decoding method that embeds a compressed model within selecte