Workshop on crossmodality and multimodality

Detailed programme:

May 23

Visual hearing: a crossmodal plasticity of the brain

11:00: Opening remarks: Notes on crossmodality and multimodality (László

Hunyadi)

1. Audio-visual transcoding: a visual aid assisting the deaf to acquire speech. Theoretical preliminaries and engineering insights

 11:05- 12:00:

- László Czap (University of Miskolc): Preliminary Test of Audio-visual Transcoding

- Péter Barabás (University of Miskolc): Application for Videogram-based Phoneme Learning

- Miklós Gábriel Tulics (Budapest University of Technology): An Overview of Computer Technology in the Study and Teaching of Pronunciation and Prosody

 2. Experimental settings

 12:00-12:30:

- István Szekrényes: multimodal recordings of Hungarian speech sounds and syllables

- (students participating in the test experiments): Seeing and hearing.

Seeing to hear. First impressions

 12:30-13:30: Buffet lunch

 3. Plasticity of the brain

 13:30-15:00:

- Zoltán Kisvárday (University of Debrecen, Dept of Anatomy, Histology and Embriology): Grouping in the visual cortex

 - Ervin Berényi (University of Debrecen, Dept of BioMedical Laboratory and Imaging Science): Connectome and Hodology. Structure and Function

 - Attila Nagy (University of Szeged, Institute of Physiology):

Multisensory integration in the basal ganglia

 Coffee break

 4. Vision, speech and sounds. Interdisciplinary insights

 15:10-16:00:

- Laurence White (University of Plymouth, School of Psychology):

“Rhythms for bears to dance to”: Communicative function and temporal form in speech

 - László Hunyadi (University of Debrecen, Dept of General and Applied

Linguistics): Recursive grouping in vision and speech. Is sound and phoneme recognition incremental?

 16:00-17:00: Round table discussion

 May 24

Multimodality in spontaneous speech and human communication

 1. Multimodality in language technology

 10:00-11:00:

- Nick Campbell (Trinity College Dublin, Centre for Language and Communications Studies): Speech science in action: Introduction to the speech research at the Speech Communication Lab in TCD

 - Tapio Seppänen (University of Oulu, Biomedical engineering and Computer engineering Laboratory): Multimodal affective computing in Oulu

 Coffee Break

 2. Data and analysis from the HuComTech corpus 1.

 11:10-12:30:

- Anikó Borbély, András Földesi (University of Debrecen, Dept of General and Applied Linguistics): Alignment of markers of unimodal and multimodal annotation

 - Ágnes Abuczki (University of Debrecen, Dept of General and Applied

Linguistics): Disambiguation of multifunctional discourse markers

 - Ghazaleh Esfandiari, István Szekrényes (University of Debrecen, Dept of General and Applied Linguistics): Multimodal analysis of topic shift in dialogues

 12:30: Buffet lunch

 3. Data and analysis from the HuComTech corpus 2.

 13:30- 15:00:

- István Szekrényes (University of Debrecen, Dept of General and Applied

Linguistics): Automatic prosodic annotation in the HuComTech corpus

 - Eniko; Tóth, Agnes Abuczki (University of Debrecen, Dept of General and Applied Linguistics): Statistical analyses of the relation between pragmatic functions and pitch movement

 - László Hunyadi (University of Debrecen, Dept of General and Applied

Linguistics): Multimodal cues to the syntax of spontaneous speech

 15:00-16:00: Round table discussion

 16:00: Closing of the workshop