Neuromorphic Unlimited Sampling for High-Dynamic-Range Video Acquisition

Abstract

The unlimited sampling framework (USF) is a computational sensing paradigm that addresses the practical bottleneck pertaining to finite dynamic range and quantization resolution of standard analog-to-digital converters (ADCs). The essence of unlimited sampling is to capture high-dynamic range (HDR) signals using practical sensors by folding the signal within the dynamic range of the sensor, followed by leveraging computational techniques for reconstruction. In this paper, we consider unlimited sampling using neuromorphic/event-driven encoders that enable acquisition of HDR signals as they simultaneously fold the signal and keep track of the folding instants using events. At ICASSP 2024, we proposed neuromorphic unlimited sampling for bandlimited signals. Herein, we extend the theory to include signals in principal shift-invariant spaces, and show that perfect reconstruction is possible without an oversampling requirement on the ADC. Samples of the folded signal along with the events are sufficient for perfect reconstruction with sampling rates that are independent of the dynamic-range of the ADC. Within the proposed technique, a large class of smooth signals that lie outside any shift-invariant space can be accommodated with the reconstruction error decreasing as the sampling interval is reduced. On the experimental front, we consider acquisition of HDR video signals using an array of neuromorphic unlimited samplers and demonstrate accurate reconstruction. The proposed neuromorphic design for HDR video acquisition does not require oversampling of the ADC because the HDR information is indirectly captured in a compressed form using binary events.