Benchmarking Arabic Emotion Detection: Transformers Evaluation using a Unified Dataset
Publikation: Beitrag in Fachzeitschrift › Forschungsartikel › Beigetragen › Begutachtung
Beitragende
Abstract
Emotion detection in Arabic text is inadequately supported and inconsistent, with current datasets exhibiting significant variability in labeling schemes, dialect representation, annotation uniformity, and data availability. These differences, along with the predominance of datasets accessible solely as Twitter IDs and the misalignment among principal emotion theories, obstruct systematic benchmarking and the advancement of resilient Arabic emotion detection models. In this work, we provide a comprehensive, data-driven survey and analysis of Arabic emotion datasets with publicly accessible text. We examine their label distributions, linguistic characteristics, and cross-dataset similarities, revealing substantial annotation inconsistencies, class imbalance, and variability in lexical and stylistic features. Building on these findings, we propose a principled unification pipeline that maps heterogeneous labeling schemes to Ekman's six basic emotions with an additional neutral category. The resulting unified dataset contains over 31K single-labeled texts spanning diverse domains and dialects. Using this resource, we conduct an extensive evaluation of state-of-the-art Arabic and multilingual transformer models in both zero-shot and fine-tuned settings. Results show broad performance disparities across emotions, notable instability in zero-shot predictions, and a clear advantage for Arabic-language models over multilingual ones. Our study demonstrates that the unification of Arabic emotion datasets is both achievable and advantageous, establishing a replicable basis for future research and presenting an extensive benchmark for emotion detection in Arabic.
Details
| Originalsprache | Englisch |
|---|---|
| Seitenumfang | 16 |
| Fachzeitschrift | IEEE Transactions on Affective Computing |
| Publikationsstatus | Elektronische Veröffentlichung vor Drucklegung - 17 Sept. 2026 |
| Peer-Review-Status | Ja |
Schlagworte
ASJC Scopus Sachgebiete
- Software - Arabic, dataset, emotion detection, language model, transformer
- Human-computer interaction - Arabic, dataset, emotion detection, language model, transformer
Schlagwörter
- Arabic, dataset, emotion detection, language model, transformer