AIConfPaper
← Search

Guillermo Marco

5 accepted papers

2025

Bilingual Evaluation of Language Models on General Knowledge in University Entrance Exams with Minimal Contamination

Eva Sánchez Salido, Roser Morante, Julio Gonzalo, Guillermo Marco, Jorge Carrillo-de-Albornoz, Laura Plaza +5
COLING 2025main

In this article we present UNED-ACCESS 2024, a bilingual dataset that consists of 1003 multiple-choice questions of university entrance level exams in Spanish and English. Questions are originally formulated in Spanish and manually translated into English, and have not ever been publicly released, e…

Cited by 1Source ↗PDF ↗Scholar ↗
2025

Small Language Models can Outperform Humans in Short Creative Writing: A Study Comparing SLMs with Humans and LLMs

Guillermo Marco, Luz Rello, Julio Gonzalo
COLING 2025main

In this paper, we evaluate the creative fiction writing abilities of a fine-tuned small language model (SLM), BART-large, and compare its performance to human writers and two large language models (LLMs): GPT-3.5 and GPT-4o. Our evaluation consists of two experiments: (i) a human study in which 68 p…

Cited by 7Source ↗PDF ↗code ↗Scholar ↗
2025

The Reader is the Metric: How Textual Features and Reader Profiles Explain Conflicting Evaluations of AI Creative Writing

Guillermo Marco, Julio Gonzalo, Víctor Fresno
ACL 2025finding

Recent studies comparing AI-generated and human-authored literary texts have produced conflicting results: some suggest AI already surpasses human quality, while others argue it still falls short. We start from the hypothesis that such divergences can be largely explained by genuine differences in h…

Cited by 0Source ↗PDF ↗code ↗Scholar ↗
2024

A Web Portal about the State of the Art of NLP Tasks in Spanish

Enrique Amigó, Jorge Carrillo-de-Albornoz, Andrés Fernández, Julio Gonzalo, Guillermo Marco, Roser Morante +2
COLING 2024main

This paper presents a new web portal with information about the state of the art of natural language processing tasks in Spanish. It provides information about forums, competitions, tasks and datasets in Spanish, that would otherwise be spread in multiple articles and web sites. The portal consists…

Cited by 3Source ↗PDF ↗Scholar ↗
2024

Pron vs Prompt: Can Large Language Models already Challenge a World-Class Fiction Author at Creative Text Writing?

Guillermo Marco, Julio Gonzalo, M.Teresa Mateo-Girona, Ramón Del Castillo Santos
EMNLP 2024main

Are LLMs ready to compete in creative writing skills with a top (rather than average) novelist? To provide an initial answer for this question, we have carried out a contest between Patricio Pron (an awarded novelist, considered one of the best of his generation) and GPT-4 (one of the top performing…

Cited by 10Source ↗PDF ↗Scholar ↗