Deep Actor-Critic for Continuous 3D Motion Control in Mobile Relay Beamforming Networks

Abstract

The paper studies the motion control for mobile relays implementing cooperative beamforming to aid the communication between a source-destination pair. We consider an urban communication scenario, where the channels exhibit spatiotemporal correlations and thus can be learned. The relays move in a time-slotted fashion within a three-dimensional cube. During every slot, the relays beamform optimally to maximize the Signal-to-Interference+Noise Ratio (SINR) at the destination and decide their positions for the next slot. Unlike prior works that assume knowledge of channel statistics, our proposed approach is model-free. Also, typically, prior approaches assume discrete motion on the two-dimensional plane. However, as discretization introduces the curse of dimensionality, those methods do not easily extend to three-dimensional motion. We propose a model-free, continuous control actor-critic approach that can be easily applied to 2D and 3D motion with the same complexity. To address the random nature of the channel, we propose to use Sinusoidal Representation Networks (SIRENs) for value function approximation. Our approach outperforms the direct application of the State-of-the-Art continuous control algorithms for both 2D and 3D cases.