(Hybrid) Deep Language Learning: Modeling Language from Raw Speech with fiwGAN
Presenters: Gasper Begus, Stephan Meylan
Offered virtuallyT2MRB2
This workshop will introduce participants to the basics of how to develop and train realistic deep learning models of human spoken language learning (fiwGAN), as well as techniques to uncover linguistically meaningful representations in deep neural networks trained on raw speech. These deep generative neural networks can not only model speech with high fidelity from a fully unsupervised objective, but have also been shown to capture a wide range of linguistic phenomena in their latent and intermediate representations.
The goal of this workshop will be to familiarize participants with the GAN framework and the linguistic relevance of these models’ representations. Participants will learn how to train and interpret these models, allowing them to pursue their own research interests on new datasets and languages of interest. We will learn how model phonetic, phonological, morphological, and even basic syntactic and lexical semantic learning.
We will then discuss techniques to test the causal relationship between learned representation and outputs of the models and test how these techniques can model rule-based symbolic computation. We will show that these networks do not just imitate training data; rather in their internal representations, they capture the causal structure of a wide range of linguistic phenomena, including phonological rules, articulatory targets, allophonic alternations, and reduplication.
We will further discuss how to apply traditional linguistic analyses to these representations and show striking similarities between these artificial neural representations and representations of speech in the human auditory system taken from neuroimaging data.
Understanding how deep neural networks learn has consequences both for linguistic theory, cognitive science, and neuroscience as well as for building interpretable machine learning.
Keywords: Computational Modeling, Learning, Theoretical Frameworks, Phonetics, AI, Deep Learning, Neurolinguistics, Production, Productivity
Room STB 245, Mondays and Thursdays, July 24-August 7, 10:30am - 11:50am (in person and virtually)
Mondays and Thursdays
Presenters
UC Berkeley
Gasper Begus is an Associate Professor at the Department of Linguistics at UC Berkeley where he directs the Berkeley Speech and Computation Lab. He is also the Linguistics Lead at Project CETI and a Member of Berkeley's Institute of Cognitive and Brain Sciences. He graduated with a Ph.D. from Harvard. In his research, he uses human, animal, and machine intelligence to understand the inner workings of AI and to better understand what language is and how it is learned. Gasper combines machine learning and statistical models with neuroimaging and behavioral experiments to better understand how deep neural networks learn internal representations and how humans learn to speak. Gasper has worked and published on sound systems of various language families such as Indo-European, Caucasian, and Austronesian languages.
I am a computational cognitive scientist who studies how children learn language, both to support the development of machine intelligence and to understand humans' remarkable cognitive capacities. As a research scientist at MIT's Department of Brain and Cognitive Sciences, my work seeks to reverse engineer the cognitive representations and learning mechanisms underlying first language development. My work uses ideas and methods from cognitive science, natural language processing, psycholinguistics, robotics, and child development to reimagine children's early communicative development as increasingly sophisticated multi-agent action coordination. This line of research builds on doctoral work looking at the role of probabilistic inference in language learning and language processing (as characterized with Bayesian and neural network models), as well as the traces left in language structures by the need for efficient inference at multiple timescales. More broadly, I am interested in the evolutionary and ecological significance of language, especially how it enables cultural transmission of knowledge and cooperative problem solving. My research has been funded by the (U.S.) National Science Foundation, the Air Force Office of Scientific Research, DARPA, the National Institutes of Health, and the Simons Foundation.
Room STB 245, Mondays and Thursdays, July 24-August 7, 10:30am - 11:50am (in person and virtually)
Mondays and Thursdays