Seeing Lexical Tone: Head and Face Motion in Production and Perception of Cantonese Lexical Tones

Publikation: Beitrag in FachzeitschriftForschungsartikelBeigetragenBegutachtung

Beitragende

  • Denis Burnham - , Western Sydney University (Autor:in)
  • Eric Vatikiotis-Bateson - , University of British Columbia (Autor:in)
  • Adriano Vilela Barbosa - , Universidade Federal de Minas Gerais (Autor:in)
  • João Vítor Possamai de Menezes - , Professur für Sprachtechnologie und Kognitive Systeme (Autor:in)
  • Hani C Yehia - , Universidade Federal de Minas Gerais (Autor:in)
  • Rua Haszard Morris - , ADInstruments (Autor:in)
  • Guillaume Vignali - , Western Sydney University (Autor:in)
  • Jessica Reynolds - , New South Wales Government (Autor:in)

Abstract

Previous studies show that lexical tones can be discriminated visually, but the locus of this information is unknown. Here we investigate the role of visual face and head information in the production and perception of the six Cantonese lexical tones. Experiment 1 (speech production) showed that tone category membership is predicted independently by each of three visual components, Head-only, Face-only, and combined Face&Head motion, that Face&Head combined is a better predictor than the sum of Head-only and Face-only, that Head-only was a better predictor than Face-only, and that up-down head motion may be involved. In Experiment 2 (speech perception) discrimination of Cantonese tones and phones (consonants and vowels) was investigated in Auditory-Only (AO), Visual-Only (VO), and Auditory-Visual (AV) presentations of Face-only, Head-only or combined Face&Head information. There was successful visual discrimination of both phones and tones, but in different manners. For VO, face movement (Face-only, Face&Head) was sufficient for Phone discrimination, whereas both head and face (Face&Head) were necessary for Tone discrimination; and Phone discrimination was better for Face-only than Head-only, whereas Tone discrimination was the better for Head-only than Face-only. For visual augmentation of auditory information, (AV-AO)/AO), for Phones there was significant facilitation in Head-only and Face-only, but not Face&Head, whereas for Tones there was significant facilitation only in the Face&Head condition. Overall, head motion is integrally involved in the differential production of Cantonese tones and, as for consonants and vowels, there is visual perception of lexical tone, albeit based on entirely different cues: face motion is sufficient for phone perception (head motion is not necessary), whereas head motion is necessary but not sufficient for tone perception (Face&Head required). Head motion is integral both to the differential production of Cantonese tones and to the visual discrimination of those tones.

Details

OriginalspracheEnglisch
Seiten (von - bis)40-55
Seitenumfang16
FachzeitschriftSpeech communication
Jahrgang141
PublikationsstatusVeröffentlicht - Apr. 2022
Peer-Review-StatusJa

Externe IDs

Scopus 85130718124

Schlagworte

Forschungsprofillinien der TU Dresden

Fächergruppen, Lehr- und Forschungsbereiche, Fachgebiete nach Destatis