Claim2Source at CheckThat! 2025: Zero-Shot Style Transfer for Scientific Claim-Source Retrieval

Research output: Contribution to book/Conference proceedings/Anthology/ReportConference contributionContributedpeer-review

Abstract

In this paper, we present our participation in the CheckThat! 2025 Task 4b on scientific claim-source retrieval. Our work systematically explores the impact of style transfer on performance in retrieving the scientific publication referenced by a COVID-19-related tweet. We apply seven distinct style transfer methods, distributed across claims and sources, to assess their impact on retrieval performance. These style transfer methods are evaluated across 15 retrieval systems, including 1 sparse, 7 dense, and 7 hybrid models, by testing each system with all combinations of claim and source styles. To guide the style transfer process, we employ a modular zero-shot prompting template with detailed instructions using a large language model (LLM). Our results show that GritLM-7B achieves the best performance without style transfer, suggesting strong robustness to informal text. In contrast, the majority of models, especially sparse and hybrid ones, benefit from applying a formal writing style to claims. We observe that hybrid retrieval models tend to outperform their dense counterparts. This highlights the potential advantage of integrating sparse and dense retrieval paradigms for scientific claim-source retrieval.

Details

Original languageEnglish
Title of host publicationCLEF 2025 Working Notes
Pages1203-1216
Number of pages14
Publication statusPublished - 2025
Peer-reviewedYes

Publication series

SeriesCEUR Workshop Proceedings
Volume4038
ISSN1613-0073

Conference

Title16th Conference and Labs of the Evaluation Forum
SubtitleInformation Access Evaluation meets Multilinguality, Multimodality, and Visualization
Abbreviated titleCLEF 2025
Conference number16
Duration9 - 12 September 2025
Website
LocationUniversidad Nacional de Educación a Distancia (UNED)
CityMadrid
CountrySpain

External IDs

ORCID /0000-0001-5458-8645/work/200631675

Keywords

ASJC Scopus subject areas

Keywords

  • Information Retrieval, Large Language Model, Text Style Transfer