Cultural commonsense knowledge for intercultural dialogues

Research output: Contribution to conferencesPaperContributedpeer-review

Contributors

Abstract

Despite recent progress, large language models (LLMs) still face the challenge of appropriately reacting to the intricacies of social and cultural conventions. This paper presents Mango, a methodology for distilling high-accuracy, high-recall assertions of cultural knowledge. We judiciously and iteratively prompt LLMs for this purpose from two entry points, concepts and cultures. Outputs are consolidated via clustering and generative summarization. Running the Mango method with GPT-3.5 as underlying LLM yields 167K high-accuracy assertions for 30K concepts and 11K cultures, surpassing prior resources by a large margin in quality and size. In an extrinsic evaluation for intercultural dialogues, we explore augmenting dialogue systems with cultural knowledge assertions. Notably, despite LLMs inherently possessing cultural knowledge, we find that adding knowledge from Mango improves the overall quality, specificity, and cultural sensitivity of dialogue responses, as judged by human annotators. Data and code are available for download.

Details

Original languageEnglish
Pages1774-1784
Number of pages11
Publication statusPublished - 21 Oct 2024
Peer-reviewedYes

Conference

Title33rd ACM International Conference on Information and Knowledge Management
Abbreviated titleCIKM 2024
Conference number33
Duration21 - 25 October 2024
Website
Degree of recognitionInternational event
LocationBoise Centre
CityBoise
CountryUnited States of America

External IDs

ORCID /0000-0002-5410-218X/work/173517501
Scopus 85209999622

Keywords

Keywords

  • cultural commonsense knowledge, intercultural dialogues, knowledge distillation