← Search

Giovanni Puccetti

7 accepted papers

2025

Optimizing LLMs for Italian: Reducing Token Fertility and Enhancing Efficiency Through Vocabulary Adaptation

NAACL 2025findings

The number of pretrained Large Language Models (LLMs) is increasing steadily, though the majority are designed predominantly for the English language. While state-of-the-art LLMs can handle other languages, due to language contamination or some degree of multilingual pretraining data, they are not o…

2025

Scaling Laws for Robust Comparison of Open Foundation Language-Vision Models and Datasets

NeurIPS 2025poster

In studies of transferable learning, scaling laws are obtained for various important foundation models to predict their properties and performance at larger scales. Taking language-vision learning as example, we show here how scaling law derivation can also be used for model and dataset comparison,…

Cited by 0SourcecodeScholar
2025

Stress-testing Machine Generated Text Detection: Shifting Language Models Writing Style to Fool Detectors

ACL 2025finding

Recent advancements in Generative AI and Large Language Models (LLMs) have enabled the creation of highly realistic synthetic content, raising concerns about the potential for malicious use, such as misinformation and manipulation. Moreover, detecting Machine-Generated Text (MGT) remains challenging…

2025

The Invalsi Benchmarks: measuring the Linguistic and Mathematical understanding of Large Language Models in Italian

COLING 2025main

While Italian is a high-resource language, there are few Italian-native benchmarks to evaluate generative Large Language Models (LLMs) in this language. This work presents three new benchmarks: Invalsi MATE to evaluate models performance on mathematical understanding in Italian, Invalsi ITA to evalu…

Cited by 0SourcePDFScholar
2024

AI ‘News’ Content Farms Are Easy to Make and Hard to Detect: A Case Study in Italian

ACL 2024long

Large Language Models (LLMs) are increasingly used as ‘content farm’ models (CFMs), to generate synthetic text that could pass for real news articles. This is already happening even for languages that do not have high-quality monolingual LLMs. We show that fine-tuning Llama (v1), mostly trained on E…

2024

M4GT-Bench: Evaluation Benchmark for Black-Box Machine-Generated Text Detection

ACL 2024long

The advent of Large Language Models (LLMs) has brought an unprecedented surge in machine-generated text (MGT) across diverse channels. This raises legitimate concerns about its potential misuse and societal implications. The need to identify and differentiate such content from genuine human-generate…

2022

Outlier Dimensions that Disrupt Transformers are Driven by Frequency

EMNLP 2022finding

While Transformer-based language models are generally very robust to pruning, there is the recently discovered outlier phenomenon: disabling only 48 out of 110M parameters in BERT-base drops its performance by nearly 30% on MNLI. We replicate the original evidence for the outlier phenomenon and we l…