Maxim K. Surkov

Vygotsky Distance: Measure for Benchmark Task Similarity

Evaluation plays a significant role in modern natural language processing. Most modern NLP benchmarks consist of arbitrary sets of tasks that neither guarantee any generalization potential for the model once applied outside the test set nor try to minimize the resource consumption needed for model e…