Learning Coherent Representations: A Topological Approach to Interpretability

Abstract

Deep neural networks learn representations where individual features often lack interpretable meaning; a single neuron may activate for scattered, unrelated inputs. We introduce coherence, a geometric property inspired by neural coding in the brain, where neurons like grid cells and head direction cells respond to contiguous regions of state space. A non-negative matrix is coherent if both each row (sample) attends to geometrically clustered columns (features) and, vice versa, each feature attends to clustered samples. We prove that coherent matrices induce a bounded interleaving between the Vietoris-Rips filtrations of samples and features, guaranteeing that both spaces share compatible topological structure. This geometric constraint facilitates interpretability. For example, if data lies on a circle, coherent features must tile that circle into contiguous arcs. We introduce COH, a differentiable regularizer based on Fréchet variance that enforces coherence during training. Unlike sparsity, which bounds how many samples a feature activates on, coherence bounds *which* samples, requiring geometric connectivity rather than only rarity. This yields not just interpretable features but an interpretable feature space. We validate COH using synthetic and rotated MNIST datasets.