← Search

Mohith Kulkarni

2 accepted papers

2024

Multi-LogiEval: Towards Evaluating Multi-Step Logical Reasoning Ability of Large Language Models

EMNLP 2024main

As Large Language Models (LLMs) continue to exhibit remarkable performance in natural language understanding tasks, there is a crucial need to measure their ability for human-like multi-step logical reasoning. Existing logical reasoning evaluation benchmarks often focus primarily on simplistic singl…

2024

Step-by-Step Reasoning to Solve Grid Puzzles: Where do LLMs Falter?

EMNLP 2024main

Solving grid puzzles involves a significant amount of logical reasoning. Hence, it is a good domain to evaluate reasoning capability of a model which can then guide us to improve the reasoning ability of models. However, most existing works evaluate only the final predicted answer of a puzzle, witho…