← Search

Charles Stewart

6 accepted papers

2024

A Simple Interpretable Transformer for Fine-Grained Image Classification and Analysis

ICLR 2024poster

We present a novel usage of Transformers to make image classification interpretable. Unlike mainstream classifiers that wait until the last fully connected layer to incorporate class information to make predictions, we investigate a proactive approach, asking each class to search for itself in an im…

2024

BioCLIP: A Vision Foundation Model for the Tree of Life

CVPR 2024poster

Images of the natural world collected by a variety of cameras from drones to individual phones are increasingly abundant sources of biological information. There is an explosion of computational methods and tools particularly computer vision for extracting biologically relevant information from imag…

2024

Fine-Tuning is Fine, if Calibrated

NeurIPS 2024poster

Fine-tuning is arguably the most straightforward way to tailor a pre-trained model (e.g., a foundation model) to downstream applications, but it also comes with the risk of losing valuable knowledge the model had learned in pre-training. For example, fine-tuning a pre-trained classifier capable of r…

2024

Hierarchical Conditioning of Diffusion Models Using Tree-of-Life for Studying Species Evolution

ECCV 2024poster

"A central problem in biology is to understand how organisms evolve and adapt to their environment by acquiring variations in the observable characteristics or traits of species across the tree of life. With the growing availability of large-scale image repositories in biology and recent advances in…

2024

VLM4Bio: A Benchmark Dataset to Evaluate Pretrained Vision-Language Models for Trait Discovery from Biological Images

NeurIPS 2024poster

Images are increasingly becoming the currency for documenting biodiversity on the planet, providing novel opportunities for accelerating scientific discoveries in the field of organismal biology, especially with the advent of large vision-language models (VLMs). We ask if pre-trained VLMs can aid sc…

2023

Holistic Transfer: Towards Non-Disruptive Fine-Tuning with Partial Target Data

NeurIPS 2023poster

We propose a learning problem involving adapting a pre-trained source model to the target domain for classifying all classes that appeared in the source data, using target data that covers only a partial label space. This problem is practical, as it is unrealistic for the target end-users to collect…

Cited by 5SourcePDFScholar