SwiftEagle: An Advanced Open-Source, Miniaturized FPGA UAS Platform with Dual DVS/Frame Camera for Cutting-Edge Low-Latency Autonomous Algorithms

Abstract

Low-latency sensing and decision-making processing are critical requirements for the highly dynamic control and perception applications often found in Unmanned Areal Systems (UASs). Novel sensors such as Dynamic Vision Sensors (DVSs) are enhancing the pure performance of the perception component with orders of magnitude lower latency. However, they are typically not optimally integrated with the computing hardware, which effectively reduces the potential in both latency and power consumption. In addition to the non-optimal integration, low latency processing of such high data rate generating sensors is often challenging on resource-constrained platforms. Here, Field Programmable Gate Arrays (FPGAs) platforms offer a promising set of features, including low-level access to hardware and memories, as well as high-speed interfaces. On the other hand, FPGA platforms are not as popular on UASs due to their complex programming and development environments, steep initial learning curve, and challenges in achieving optimal performance. To accelerate future development, this paper presents SwiftEagle, an open-source, cutting-edge 720g FPGA based UAS based on a custom hardware design including a dual camera interface RGB/DVSs on a multi-sensors subsystem and an initial software and firmware stack designed for high precision recording of machine learning datasets in-flight with sub-micro-second time resolution and on-FPGA rendering of DVS event frames. Utilizing this developed platform, as proof-of-concept the end-to-end latency of a novel, just released DVS is shown to be below 210µs as a worst-case scenario, enabling future cutting-edge autonomous algorithms.