Preface: LM-KBC Challenge 2024

Publikation: Beitrag in Buch/Konferenzbericht/Sammelband/GutachtenVor-/NachwortBeigetragen

Beitragende

Abstract

Pretrained language models (LMs) have advanced a range of semantic tasks and have also shown promise for knowledge extraction from the models itself. Although several works have explored this ability in a setting called probing or prompting, the viability of knowledge base construction from LMs remains underexplored. In the 3rd edition of this challenge, participants were asked to build actual disambiguated knowledge bases from LMs, for given subjects and relations. In crucial difference to existing probing benchmarks like LAMA [1], we make no simplifying assumptions on relation cardinalities, i.e., a subject-entity can stand in relation with zero, one, or many object-entities. Furthermore, submissions need to go beyond just ranking predicted surface strings and materialize disambiguated entities in the output, which will be evaluated using established KB metrics of precision and recall. The challenge has a single track for LMs with a parameter level under 10-billion to fit into low computational requirements. The challenge received 8 submissions, of which 5 submitted a paper, and 4 were accepted for presentation. The challenge was collocated with a workshop on related topics, allowing to host extended discussions, related papers, and invited talks.

Details

OriginalspracheEnglisch
TitelKBC-LM-LM-KBC 2024 - Joint proceedings of the KBC-LM workshop and the LM-KBC challenge 2024
Seitenumfang5
Band3853
PublikationsstatusVeröffentlicht - 2024
Peer-Review-StatusNein

Publikationsreihe

ReiheCEUR Workshop Proceedings
Band3853
ISSN1613-0073

Workshop

Titel2nd Workshop on Knowledge Base Construction from Pre-Trained Language Models
KurztitelKBC-LM 2024
Dauer12 November 2024
Webseite
OrtLive! Casino & Hotel Maryland
StadtBaltimore
LandUSA/Vereinigte Staaten

Externe IDs

ORCID /0000-0002-5410-218X/work/173989365
Scopus 85212702735

Schlagworte

ASJC Scopus Sachgebiete