Weighted parsing for grammar-based language models over multioperator monoids
Research output: Contribution to journal › Research article › Contributed › peer-review
Contributors
Abstract
We develop a general framework for weighted parsing which is built on top of grammar-based language models and employs multioperator monoids (M-monoids) as weight algebras. It generalizes previous work in that area, e.g., semiring parsing and weighted deductive parsing, and also covers applications outside the classical scope of parsing, e.g., algebraic dynamic programming. More specifically, we introduce weighted RTG-based language models (where RTG stands for regular tree grammar) and define the M-monoid parsing problem. We show an algorithm which is supposed to solve this problem and prove in detail that, for a large class of weighted RTG-based language models, the algorithm terminates and, indeed, solves this problem. We compare our algorithm with semiring parsing and weighted deductive parsing regarding applicability and complexity.
Details
Original language | English |
---|---|
Article number | 104774 |
Journal | Information and Computation |
Volume | 281 |
Publication status | Published - Dec 2021 |
Peer-reviewed | Yes |
External IDs
Scopus | 85109440756 |
---|
Keywords
ASJC Scopus subject areas
Keywords
- Weighted parsing, Language model, Formal grammar, Formal grammar, Language model