Ethan Chau — accepted AI-conference papers

2024

Automatic Pair Construction for Contrastive Post-training

Canwen Xu, Corby Rosset, Ethan Chau, Luciano Corro, Shweti Mahajan, Julian McAuley +3

NAACL 2024findings

Alignment serves as an important step to steer large language models (LLMs) towards human preferences. In this paper, we propose an automatic way to construct contrastive data for LLM, using preference pairs from multiple models of varying strengths (e.g., InstructGPT, ChatGPT and GPT-4). We compare…

Cited by 2 Source ↗PDF ↗code ↗Scholar ↗

2024

Dodo: Dynamic Contextual Compression for Decoder-only LMs

Guanghui Qin, Corby Rosset, Ethan Chau, Nikhil Rao, Benjamin Van Durme

ACL 2024long

Transformer-based language models (LMs) are inefficient in long contexts. We propose Dodo, a solution for context compression. Instead of one vector per token in a standard transformer model, Dodo represents text with a dynamic number of hidden states at each layer, reducing the cost of self-attenti…

Cited by 7 Source ↗PDF ↗Scholar ↗