Benchmarking Arabic Emotion Detection: Transformers Evaluation using a Unified Dataset

Publikation: Beitrag in Fachzeitschrift › Forschungsartikel › Beigetragen › Begutachtung

Beitragende

Abstract

Emotion detection in Arabic text is inadequately supported and inconsistent, with current datasets exhibiting significant variability in labeling schemes, dialect representation, annotation uniformity, and data availability. These differences, along with the predominance of datasets accessible solely as Twitter IDs and the misalignment among principal emotion theories, obstruct systematic benchmarking and the advancement of resilient Arabic emotion detection models. In this work, we provide a comprehensive, data-driven survey and analysis of Arabic emotion datasets with publicly accessible text. We examine their label distributions, linguistic characteristics, and cross-dataset similarities, revealing substantial annotation inconsistencies, class imbalance, and variability in lexical and stylistic features. Building on these findings, we propose a principled unification pipeline that maps heterogeneous labeling schemes to Ekman's six basic emotions with an additional neutral category. The resulting unified dataset contains over 31K single-labeled texts spanning diverse domains and dialects. Using this resource, we conduct an extensive evaluation of state-of-the-art Arabic and multilingual transformer models in both zero-shot and fine-tuned settings. Results show broad performance disparities across emotions, notable instability in zero-shot predictions, and a clear advantage for Arabic-language models over multilingual ones. Our study demonstrates that the unification of Arabic emotion datasets is both achievable and advantageous, establishing a replicable basis for future research and presenting an extensive benchmark for emotion detection in Arabic.

Details

OriginalspracheEnglisch
Seitenumfang16
FachzeitschriftIEEE Transactions on Affective Computing
PublikationsstatusElektronische Veröffentlichung vor Drucklegung - 17 Sept. 2026
Peer-Review-StatusJa

Schlagworte

ASJC Scopus Sachgebiete

Schlagwörter

  • Arabic, dataset, emotion detection, language model, transformer