Heyi Tao — accepted AI-conference papers

2024

Region-Based Representations Revisited

Michal Shlapentokh-Rothman, Ansel Blume, Yao Xiao, Yuqun Wu, Sethuraman TV, Heyi Tao +4

CVPR 2024poster

We investigate whether region-based representations are effective for recognition. Regions were once a mainstay in recognition approaches but pixel and patch-based features are now used almost exclusively. We show that recent class-agnostic segmenters like SAM can be effectively combined with strong…

Cited by 7 Source ↗PDF ↗code ↗Scholar ↗

2024

WebWISE: Unlocking Web Interface Control for LLMs via Sequential Exploration

Heyi Tao, Sethuraman T V, Michal Shlapentokh-Rothman, Tanmay Gupta, Heng Ji, Derek Hoiem

NAACL 2024findings

This paper investigates using Large Language Models (LLMs) to automatically perform web software tasks using click, scroll, and text in- put operations. Previous approaches, such as reinforcement learning (RL) or imitation learning, are inefficient to train and task-specific. Our method uses filtere…

Cited by 4 Source ↗PDF ↗Scholar ↗