State-Space Time Surfaces for Event-Based Zero-Shot Robotic Grasping and Scene Reconstruction

Abstract

Event cameras report per-pixel brightness changes asynchronously with microsecond latency, but their output is incompatible with vision foundation models trained on conventional images. We propose State-Space Time Surfaces (S3TS), a training-free representation that recasts exponential-decay time surfaces as a diagonal state-space model with multi-scale temporal channels and Mamba-inspired selective decay. The resulting pseudo-RGB image is fed directly to a frozen OWLv2 detector for zero-shot, text-prompted object detection from events alone. We demonstrate two applications on a 6-DOF manipulator: event-only grasping with near-nadir refinement, and dense 3D scene reconstruction via multi-view TSDF fusion with neuromorphic surface descriptors. S3TS detects over twice as many objects as single-channel event representations and produces faithful 3D workspace meshes