Optical Flow Estimation Using Speck Neuromorphic Hardware

Abstract

Neuromorphic hardware and spiking neural networks (SNNs) offer a bio-inspired path to low-latency, energy-efficient computation by emulating the brain’s asynchronous, spike-based processing. This is particularly attractive for resource-constrained robots that are tightly limited in size, weight, and power. We propose a neuromorphic approach to real-time optical flow estimation tailored to the SynSense Speck system-on-chip, which integrates a Dynamic Vision Sensor (DVS) with a neuromorphic processor. Our inference architecture combines spiking and artificial neural layers in a hybrid SNN–ANN framework, enabling the use of Speck to perform regression for closed-loop drone control, an application not previously demonstrated on this chip. Despite its compact form factor, the system produces dense flow in real time and achieves stable indoor hover and forward flight using flow-based control. The hybrid pipeline runs ~2x faster than an ANN-only baseline at identical power, highlighting the promise of neuromorphic sensing and processing for ultra-efficient autonomous flight in real-world scenarios. Code and data are available at: https://mavlab.tudelft.nl/speck-optical-flow