2024
Online Cascade Learning for Efficient Inference over Streams
ICML 2024poster
Large Language Models (LLMs) have a natural role in answering complex queries about data streams, but the high computational cost of LLM inference makes them infeasible in many such tasks. We propose *online cascade learning*, the first approach to address this challenge. The objective here is to le…