The role of face and head movement in the production of lexical tones in Cantonese

Research output: Contribution to book/conference proceedings/anthology/reportConference contributionContributedpeer-review

Contributors

  • João Vítor Possamai de Menezes - , Chair of Speech Technology and Cognitive Systems (Author)
  • Maria Mendes Cantoni - , Universidade Federal de Minas Gerais (Author)
  • Hani C Yehia - , Universidade Federal de Minas Gerais (Author)
  • Denis Burnham - , Western Sydney University (Author)
  • Adriano Vilela Barbosa - , Universidade Federal de Minas Gerais (Author)

Abstract

Speech is a multimodal phenomenom at the perception and pro-
duction ends, and that includes the suprasegmental level of
speech. This paper focuses on the auditory-visual nature of lex-
ical tones, a suprasegmental unit of speech that characterises
tone languages. A multimodal corpus consisting of audio and
Optotrak recordings of 33 markers in the face and head was
recorded with 3 native speakers of Cantonese. The recorded tra-
jectories of the Optotrak markers were parameterized as poly-
nomial coefficients and used as input to Linear Discriminant
Analysis models for classification between the 6 Cantonese lex-
ical tones. Face and head motion were able to classify between
lexical tones with above-chance accuracy for each speaker in-
dividually and for all speakers combined. Other analyses were
carried out to determine which face regions and types of head
motion had a stronger influence of the lexical tone classification
accuracy, and the movement of the eyebrows and of the larynx
stood out.

Details

Original languageEnglish
Title of host publicationProc. ISSP 2024 - 13th International Seminar on Speech Production
Pages27-30
Number of pages4
Publication statusPublished - 2024
Peer-reviewedYes

External IDs

ORCID /0000-0002-7612-9754/work/166324334
unpaywall 10.21437/issp.2024-8

Keywords

Research priority areas of TU Dresden

Subject groups, research areas, subject areas according to Destatis