7 accepted papers
Recently, quantization has been widely used for the compression and acceleration of large language models (LLMs). Due to the outliers in LLMs, it is crucial to flatten weights and activations to minimize quantization error with equally spaced quantization points. Prior research explores various pre-…
Large language models (LLMs) excel in natural language processing but demand intensive computation. To mitigate this, various quantization methods have been explored, yet they compromise LLM performance. This paper unveils a previously overlooked type of outliers in LLMs. Such outliers are found to…
Simplex component analysis (SCA) aims to estimate the vertices of the convex hull where data samples reside in. SCA finds various applications in signal processing, e.g., hyperspectral unmixing and noisy label learning. Recent works proposed to tackle SCA from a probabilistic viewpoint using variati…
In this paper we develop a simple scheme for a coupled matrix factorization problem arising in the topic of hyperspectral super-resolution (HSR). HSR considers the problem of recovering a super-resolution image from a multispectral image and a hyperspectral image, which have lower spectral and spati…
Policy distillation, which transfers a teacher policy to a student policy has achieved great success in challenging tasks of deep reinforcement learning. This teacher-student framework requires a well-trained teacher model which is computationally expensive. Moreover, the performance of the student…
Hyperspectral unmixing (HU) is a problem of blindly identifying the underlying materials, in form of spectral signatures, in the captured hyperspectral image. HU has received tremendous interest in remote sensing, and fundamentally the problem can be regarded as solving a simplex-structured matrix f…
Graph neural networks (GNNs), which learn the representation of a node by aggregating its neighbors, have become an effective computational tool in downstream applications. Over-smoothing is one of the key issues which limit the performance of GNNs as the number of layers increases. It is because th…