21 accepted papers
Deep predictive coding networks (DPCNs) effectively model and capture video features through a bi-directional inference without labels. They are based on an overcomplete description of video scenes, and one of the bottlenecks has been the lack of effective sparsification techniques to find discrimin…
In this paper, we propose the Schoenberg kernel loss, a loss function that measures the divergence between spike trains for spiking neural network training. Our method addresses the challenges of non-differentiable spiking neurons and leverages both spatial and temporal dynamics in information proce…
Training on extremely long sequences poses significant challenges for attention mechanisms. In this paper, we introduce a novel trim attention mechanism that capitalizes on the inherent sparsity within attention processes. This mechanism effectively compresses the sequence length, thereby reducing t…
We develop a new neural network based independent component analysis (ICA) method by directly minimizing the dependence amongst all extracted components. Using the matrix-based Rényi’s α-order entropy functional, our network can be directly optimized by stochastic gradient descent (SGD), without any…
We introduce the matrix-based Rényi’s α-order entropy functional to parameterize Tishby et al. information bottleneck (IB) principle [1] with a neural network. We term our methodology Deep Deterministic Information Bottleneck (DIB), as it avoids variational inference and distribution assumption. We…
This paper presents a novel training methodology to adapt parameters of a bank of Wiener models (BWMs), i.e., a bank of linear filters followed by a static memoryless nonlinearity, using full pdf information of the projected outputs and the desired signal. BWMs also share the same architecture with…
We demonstrate that a systematic inclusion of prior structural constraints on the states of a linear dynamical system significantly improves its ability to model complex multidimensional sequences. This constrained LDS, typically termed as the hierarchical linear dynamical system (HLDS), is a Kalman…
In this paper, we address the problem of estimating a scene's three-dimensional geometry from plenoptic camera images. Existing approaches for this problem have emphasized the development of sharpness and contrast measures for distinguishing between in-/out-of-focus image regions. The ways in which…
A fundamental problem when aggregating Markov chains is the specification of the number of state groups. Too few state groups may fail to sufficiently capture the pertinent dynamics of the original, high-order Markov chain. Too many state groups may lead to a non-parsimonious, reduced-order Markov c…
We propose a novel cascaded classification technique called the Nearest Instance Centroid Estimation (NICE) LDA algorithm. Our algorithm (inspired from NICE KLMS) performs a cascade combination of two weak classifiers - threshold based class-wise clustering and linear discriminant classification to…
In this paper, we provide an approach to clustering relational matrices whose entries correspond to either similarities or dissimilarities between objects. Our approach is based on the value of information, a parameterized, information-theoretic criterion that measures the change in costs associated…
Cognition emerges from complex interaction amongst widespread brain areas. In this paper, we use a novel methodology for temporal networks quantification for EEG. We model the spatiotemporal structure of dependencies across different electrodes with respect to a single electrode as a local probabili…
It is almost seventy years after the publication of Claude Shannon's “A Mathematical Theory of Communication” [1] and Norbert Wiener's “Extrapolation, Interpolation and Smoothing of Stationary Time Series” [2]. The pioneering works of Shannon and Wiener lay the foundation of communication, data stor…
Automatic insect recognition (AIR), using noninvasive methods in situ, has far-reaching implications in entomology, agriculture, and disease control and prevention. An emerging technology in computational entomology uses flight information captured by laser sensors. Current methods treat these optic…
In this paper, we consider an information-theoretic approach for addressing the exploration-exploitation dilemma in reinforcement learning. We employ the value of information, a criterion that provides the optimal trade-off between the expected returns and a policy's degrees of freedom. As the degre…
This paper proposes a way of enhancing shape recognition through point set registration. Firstly, a modified version of shape context (SC) is developed, which is invariant to rigid transformation and flipping. With the point correspondence obtained by the modified SC, an affine transformation based…
Saliency measures are a popular way to predict visual attention. However, saliency is normally tested on sets of single resolution images that are unlike what the human vision system sees. We propose a new saliency measure based on convolving images with 2D gamma kernels which function as a comparis…
We introduce a novel, transient model for the electroencephalogram (EEG) as the noisy addition of linear filters responding to trains of delta functions. We set the synthesis part as a parameter-tuning problem and obtain synthetic EEG-like data that visually resembles brain activity in the time and…
Sparsely-activated time series are found in many physical systems. In these cases, the signals can be approximated by convolution of sparse sources with a set of shift-invariant filters. When there is access to only one sensor, such that there is a single observation signal, identifying the source s…
In this paper we investigate the advantages of learning representations of color plus depth images (Red-Blue-Green-Depth, RGB-D) over color only images (RGB) for computer vision. Specifically, we investigate the advantages on the task of object recognition. For this purpose, we applied the state-of-…
Sparse estimation has received a lot of attention due to its broad applicability. In sparse channel estimat ion, the parameter vector with sparsity characteristic can be well estimated from noisy measurements through sparse adaptive filters. In previous studies, most works use the mean square error…