Estimation of Information in Parallel Gaussian Channels via Model Order Selection
Carlos Alejandro López, Ferran De Cabrera, Jaume Riba
Abstract
We study the problem of estimating the overall mutual information in M independent parallel discrete-time memory-less Gaussian channels from N independent data sample pairs per channel (inputs and outputs). We focus on the case where the number of active channels L is sparse in comparison with the total number of channels (L ≪ M), for which the direct application of the maximum likelihood principle is problematic due to overfitting, especially for moderate to small N. For this regime, we show that the bias of the mutual information estimate is reduced by resorting to the minimum description length (MDL) principle. As a result, simple pre-processing based on a per-channel threshold on the empirical squared correlation coefficient is required with a fixed threshold that monotonically decreases with N as 1 - N <sup xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">-1/N</sup> , for N ≥ 4. The resulting improvement is shown in terms of the estimated information bias.
BibTeX
@inproceedings{icassp2020_estimationofinfo,
title = {Estimation of Information in Parallel Gaussian Channels via Model Order Selection},
author = {Carlos Alejandro López and Ferran De Cabrera and Jaume Riba},
booktitle = {ICASSP 2020},
year = {2020}
}