← Search

Shyam Upadhyay

6 accepted papers

2026

Vibe Checker: Aligning Code Evaluation with Human Preference

ICML 2026poster

Large Language Models (LLMs) have catalyzed vibe coding, where users leverage LLMs to generate and iteratively refine code through natural language interactions until it passes their *vibe check*. *Vibe check* reflects human preference and goes beyond functionality: the solution should feel right, r…

Cited by 0SourceScholar
2025

Fact, Fetch, and Reason: A Unified Evaluation of Retrieval-Augmented Generation

NAACL 2025long

Large Language Models (LLMs) have demonstrated significant performance improvements across various cognitive tasks. An emerging application is using LLMs to enhance retrieval-augmented generation (RAG) capabilities. These systems require LLMs to understand user queries, retrieve relevant information…

Cited by 15SourcePDFScholar
2024

AutoMix: Automatically Mixing Language Models

NeurIPS 2024poster

Large language models (LLMs) are now available from cloud API providers in various sizes and configurations. While this diversity offers a broad spectrum of choices, effectively leveraging the options to optimize computational cost and performance remains challenging. In this work, we present AutoMi…

2022

TableFormer: Robust Transformer Modeling for Table-Text Encoding

ACL 2022long

Understanding tables is an important aspect of natural language understanding. Existing models for table understanding require linearization of the table structure, where row or column order is encoded as an unwanted bias. Such spurious biases make the model vulnerable to row and column order pertur…

2021

TIMEDIAL: Temporal Commonsense Reasoning in Dialog

ACL 2021long

Everyday conversations require understanding everyday events, which in turn, requires understanding temporal commonsense concepts interwoven with those events. Despite recent progress with massive pre-trained language models (LMs) such as T5 and GPT-3, their capability of temporal reasoning in dialo…

2018

(Almost) Zero-Shot Cross-Lingual Spoken Language Understanding

ICASSP 2018accepted

Spoken language understanding (SLU) is a component of goal-oriented dialogue systems that aims to interpret user's natural language queries in system's semantic representation format. While current state-of-the-art SLU approaches achieve high performance for English domains, the same is not true for…

Cited by 0SourceScholar