Roberto L. Castro — accepted AI-conference papers

2026

Bridging the Gap Between Promise and Performance for FP4 Quantization

Vage Egiazarian, Roberto L. Castro, Denis Kuznedelev, Andrei Panferov, Saleh Ashkboos, Eldar Kurtic +5

ICLR 2026poster

The recent hardware-accelerated microscaling 4-bit floating-point formats such as MXFP4 and NVFP4, supported on NVIDIA and AMD GPUs, promise to revolutionize large language model (LLM) inference. Yet, their practical benefits remain unproven. We present the first comprehensive study of MXFP4 and NV…

Cited by 0 Source ↗code ↗Scholar ↗

2025

HALO: Hadamard-Assisted Lower-Precision Optimization for LLMs

Saleh Ashkboos, Mahdi Nikdan, Soroush Tabesh, Roberto L. Castro, Torsten Hoefler, Dan Alistarh

NeurIPS 2025poster

Quantized training of Large Language Models (LLMs) remains an open challenge, as maintaining accuracy while performing all matrix multiplications in low precision has proven difficult. This is particularly the case when fine-tuning pre-trained models, which can have large weight, activation, and err…

Cited by 0 Source ↗code ↗Scholar ↗

2025

Quartet: Native FP4 Training Can Be Optimal for Large Language Models

Roberto L. Castro, Andrei Panferov, Soroush Tabesh, Oliver Sieberling, Jiale Chen, Mahdi Nikdan +2

NeurIPS 2025poster

Training large language models (LLMs) models directly in low-precision offers a way to address computational costs by improving both throughput and energy efficiency. For those purposes, NVIDIA's recent Blackwell architecture facilitates very low-precision operations using FP4 variants. Yet, curren…

Cited by 0 Source ↗code ↗Scholar ↗