Embedded Named Entity Recognition using Probing Classifiers
Publikation: Beitrag in Buch/Konferenzbericht/Sammelband/Gutachten › Beitrag in Konferenzband › Beigetragen › Begutachtung
Beitragende
Abstract
Streaming text generation, has become a common way of increasing the responsiveness of language model powered applications such as chat assistants. At the same time, extracting semantic information from generated text is a useful tool for applications such as automated fact checking or retrieval augmented generation. Currently, this requires either separate models during inference, which increases computational cost, or destructive fine-tuning of the language model. Instead, we propose an approach called EMBER which enables streaming named entity recognition in decoder-only language models without fine-tuning them and while incurring minimal additional computational cost at inference time. Specifically, our experiments show that EMBER maintains high token generation rates, with only a negligible decrease in speed of around 1% compared to a 43.64% slowdown measured for a baseline. We make our code and data available online, including a toolkit for training, testing, and deploying efficient token classification models optimized for streaming text generation.
Details
Originalsprache | Englisch |
---|---|
Titel | "Proceedings of the 2024 Conference on Empirical Methods in Natural Language Processing |
Erscheinungsort | Miami, Florida, USA |
Herausgeber (Verlag) | Association for Computational Linguistics (ACL) |
Seiten | 17830-17850 |
Seitenumfang | 21 |
Publikationsstatus | Veröffentlicht - Nov. 2024 |
Peer-Review-Status | Ja |
Konferenz
Titel | 2024 Conference on Empirical Methods in Natural Language Processing |
---|---|
Kurztitel | EMNLP 2024 |
Dauer | 12 - 16 November 2024 |
Webseite | |
Bekanntheitsgrad | Internationale Veranstaltung |
Ort | Hyatt Regency Miami Hotel & Online |
Stadt | Miami |
Land | USA/Vereinigte Staaten |