NAIL: Lexical Retrieval Indices with Efficient Non-Autoregressive Decoders
Neural document rerankers are extremely effective in terms of accuracy. However, the best models require dedicated hardware for serving, which is costly and often not feasible. To avoid this servingtime requirement, we present a method of capturing up to 86% of the gains of a Transformer cross-atten…