2025
C3PO: Optimized Large Language Model Cascades with Probabilistic Cost Constraints for Reasoning
NeurIPS 2025poster
Large language models (LLMs) have achieved impressive results on complex reasoning tasks, but their high inference cost remains a major barrier to real-world deployment. A promising solution is to use cascaded inference, where small, cheap models handle easy queries, and only the hardest examples ar…