Mixed Spiking NeRF: Towards a More Efficient Neural Radiance Fields

Abstract

In recent years, 3D reconstruction has garnered significant attention. Traditional Neural Radiance Fields (NeRF) can generate high-quality, photorealistic 3D models, rendering object surfaces and texture details through computer vision tasks. However, traditional NeRF requires significant computational resources and extensive training time To further reduce power consumption and hardware requirements, this paper introduces a brain-inspired neural network mechanism aimed at optimizing low-power 3D reconstruction tasks and proposes a hybrid adaptive Spiking Neural Network (SNN) NeRF framework called Mixed Spiking NeRF. Spiking Neural Networks excel in handling sparse data, implementing information augmentation in the form of binary images, and utilizing Leaky Integrate-and-Fire (LIF) neurons to promote efficient communication across regions. In this work, we integrate the Group Neurons (GN) mechanism into the framework. We introduce comprehensive GN operations, including both horizontal and vertical GN, to enable efficient patch communication. Finally, through optimal group-based loss function optimization, across various datasets and scenarios, our framework achieved impressive results that indicate the energy consumption is reduced by 60%.