Claim2Source at CheckThat! 2025: Zero-Shot Style Transfer for Scientific Claim-Source Retrieval

Publikation: Beitrag in Buch/Konferenzbericht/Sammelband/GutachtenBeitrag in KonferenzbandBeigetragenBegutachtung

Abstract

In this paper, we present our participation in the CheckThat! 2025 Task 4b on scientific claim-source retrieval. Our work systematically explores the impact of style transfer on performance in retrieving the scientific publication referenced by a COVID-19-related tweet. We apply seven distinct style transfer methods, distributed across claims and sources, to assess their impact on retrieval performance. These style transfer methods are evaluated across 15 retrieval systems, including 1 sparse, 7 dense, and 7 hybrid models, by testing each system with all combinations of claim and source styles. To guide the style transfer process, we employ a modular zero-shot prompting template with detailed instructions using a large language model (LLM). Our results show that GritLM-7B achieves the best performance without style transfer, suggesting strong robustness to informal text. In contrast, the majority of models, especially sparse and hybrid ones, benefit from applying a formal writing style to claims. We observe that hybrid retrieval models tend to outperform their dense counterparts. This highlights the potential advantage of integrating sparse and dense retrieval paradigms for scientific claim-source retrieval.

Details

OriginalspracheEnglisch
TitelCLEF 2025 Working Notes
Seiten1203-1216
Seitenumfang14
PublikationsstatusVeröffentlicht - 2025
Peer-Review-StatusJa

Publikationsreihe

ReiheCEUR Workshop Proceedings
Band4038
ISSN1613-0073

Konferenz

Titel16th Conference and Labs of the Evaluation Forum
UntertitelInformation Access Evaluation meets Multilinguality, Multimodality, and Visualization
KurztitelCLEF 2025
Veranstaltungsnummer16
Dauer9 - 12 September 2025
Webseite
OrtUniversidad Nacional de Educación a Distancia (UNED)
StadtMadrid
LandSpanien

Externe IDs

ORCID /0000-0001-5458-8645/work/200631675

Schlagworte

ASJC Scopus Sachgebiete

Schlagwörter

  • Information Retrieval, Large Language Model, Text Style Transfer