Embedded Named Entity Recognition using Probing Classifiers

Publikation: Beitrag in Buch/Konferenzbericht/Sammelband/GutachtenBeitrag in KonferenzbandBeigetragenBegutachtung

Abstract

Streaming text generation, has become a common way of increasing the responsiveness of language model powered applications such as chat assistants. At the same time, extracting semantic information from generated text is a useful tool for applications such as automated fact checking or retrieval augmented generation. Currently, this requires either separate models during inference, which increases computational cost, or destructive fine-tuning of the language model. Instead, we propose an approach called EMBER which enables streaming named entity recognition in decoder-only language models without fine-tuning them and while incurring minimal additional computational cost at inference time. Specifically, our experiments show that EMBER maintains high token generation rates, with only a negligible decrease in speed of around 1% compared to a 43.64% slowdown measured for a baseline. We make our code and data available online, including a toolkit for training, testing, and deploying efficient token classification models optimized for streaming text generation.

Details

OriginalspracheEnglisch
Titel"Proceedings of the 2024 Conference on Empirical Methods in Natural Language Processing
ErscheinungsortMiami, Florida, USA
Herausgeber (Verlag)Association for Computational Linguistics (ACL)
Seiten17830-17850
Seitenumfang21
PublikationsstatusVeröffentlicht - Nov. 2024
Peer-Review-StatusJa

Konferenz

Titel2024 Conference on Empirical Methods in Natural Language Processing
KurztitelEMNLP 2024
Dauer12 - 16 November 2024
Webseite
BekanntheitsgradInternationale Veranstaltung
OrtHyatt Regency Miami Hotel & Online
StadtMiami
LandUSA/Vereinigte Staaten

Schlagworte

Forschungsprofillinien der TU Dresden