Individualized feedback in computer-assisted spoken language learning

• Applicant: Professor Dr. Bernd Möbius
• Subject Area: General and Comparative Linguistics, Experimental Linguistics, Typology, Non-European Languages
• Term: from 2013 to 2016
• Project identifier: Deutsche Forschungsgemeinschaft (DFG) - Project number 230875573

• Final Report Year: 2017

Final Report Abstract

IFCASL (Individualized Feedback for Computer-Assisted Spoken Language Learning) is a project conducted in partnership between Saarland University in Saarbrücken, Germany and LORIA in Nancy, France. The aim of the project was to improve the usability and efficiency of computer-assisted language learning software by adapting content, feedback and exercises to individual learners in the speech modality of a foreign language. This was achieved by integrating the production and perception of the learner’s own speech and focusing on the French-German language pair, in both directions. Providing learners with feedback is a central issue of foreign language learning. Therefore, the first objective was to provide the learner with feedback derived from an analysis of the learner’s utterance and targeting specifically the acoustic features that need to be improved. The second objective was to offer feedback that relies on phonetic knowledge incorporated in the learning system and interacting with automatic speech recognition (ASR) and signal processing (WP 4/5) methods. The third objective was to reach a high level of individualization of the user interface, type of exercises, nature and level of feedback (textual, acoustic, visual) and to develop a learner profile, taking into account the phonetic difficulties experienced by the individual learner on the levels of production and perception (WP 4). This objective required a precise definition of the phonetic difficulties predicted for the L1–L2 pair by speech scientists of both languages. Difficulties concern the segmental level (e.g., stop consonant aspiration, final obstruent devoicing, vowel quantity contrasts) and the suprasegmental level (e.g., lexical stress, contrastive accent). The target features were selected by considering the phonological and phonetic systems of both languages and ranked by observed recurring errors made by learners, to ensure a relevant learning progress. This procedure was driven by a careful analysis of a corpus comprising French and German L2 speech data recorded by learners at different levels of proficiency (WP 1/2/3). The IFCASL corpus, comprising the speech recordings and their annotation on various phonetic and linguistic levels, is currently being integrated into the Clarin-D repository, which will provide a professional framework for data sustainability and for making the corpus available to the scientific community. A snapshot of the corpus illustrating L1–L2 interference phenomena and selected levels of phonetic annotation is provided at http://www.ifcasl.org/corpus.html.

Publications

• 2014. Too cautious to vary more? A comparison of pitch variation in native and non-native productions of French and German speakers. Proc. Speech Prosody (SP7), Dublin, pp. 1037–1041
  Zimmerer, F., Jügler, J., Andreeva, B., Möbius, B. & Trouvain, J.
  
• (2015). The effect of high variability training on the perception and production of French stops by German native speakers. Interspeech 2015, Dresden, pp. 806–810
  Jügler, J., Zimmerer, F., Möbius, B. & Draxler, C.
  
• 2015. Analysis of phone confusion matrices in a manually annotated French-German learner corpus. Workshop on Speech and Language Technology in Education (SLaTE 2015), Leipzig, pp. 107–112
  Jouvet, D., Bonneau, A., Trouvain, J., Zimmerer, F., Laprie, Y. & Möbius, B.
  
• 2015. Auditory feedback methods to improve the pronunciation of stops by German learners of French. 18th International Congress of Phonetic Sciences (ICPhS), Glasgow, pp. 1–5
  Jügler, J. & Möbius, B.
  
• 2016. Breath and non-breath pauses in fluent and disfluent phases of German and French L1 and L2 Read Speech. Proceedings 8th Conference on Speech Prosody, Boston, pp. 31–35
  Trouvain, J., Fauth, C. & Möbius, B.
  (See online at https://dx.doi.org/10.21437/SpeechProsody.2016-7)
• 2016. Influence of L1 prominence on L2 production. Proceedings 8th Conference on Speech Prosody, Boston, pp. 370–374
  Zimmerer, F., Bonneau, A. & Andreeva, B.
  
• 2016. Phonetische Lernerkorpora und ihr Nutzen im DaF- Bereich. Deutsch als Fremdsprache 53(4), pp. 204–212
  Trouvain, J. & Zimmerer, F.
  
• 2016. The IFCASL corpus of French and German nonnative and native read speech. Proceedings 9th Language Resources and Evaluation Conference (LREC), Portorož, pp. 1333–1338
  Trouvain, J., Bonneau, A., Colotte, V., Fauth, C., Fohr, D., Jouvet, D., Jügler, J., Laprie, Y., Mella, O., Möbius, B. & Zimmerer, F.
  
• 2016. The perceptual effect of L1 prosody transplantation on L2 speech: The case of French accented German. Proceedings of Interspeech 2016, San Francisco, pp. 67–71
  Jügler, J., Zimmerer, F., Trouvain, J. & Möbius, B.
  (See online at https://dx.doi.org/10.21437/Interspeech.2016-1268)