Information Theory Tools for Language Research

Presenters: Fermin Moscoso del Prado

BMRD1

This workshop meets 9 times. Information Theory is a powerful set of concepts and methods that extend the power of Probability Theory. At its origin, Information Theory was developed for the study of communication over transmission lines, and was readily taken up as a powerful tool by physicists and electrical engineers. Despite its origins in the study of communication, it is only in the last two decades that the application of such tools and methods has become frequent in linguistic research. However, its use is now becoming widespread in research covering many sub-areas of linguistics, including, but not restricted to: phonology/phonetics, morphology, syntax, historical linguistics, and language acquisition. This workshop will present an introduction to the concepts and tools of information theory specifically designed for linguistics attendees. In particular we will explore the concepts of information and entropy, and the methods for their estimation from data. I will divide the study of Information Theory into two broad blocks:
- Static Methods, roughly corresponding to the paradigmatic aspects of linguistics, including estimations of diversity of linguistic constructions.
- Dynamic Methods, roughly matching the syntagmatic aspects of linguistics, including the study of predictability in sequences.

Keywords: Information Theory, Probabilistic Models, Quantitative Methods, Computational Modeling

Presenters

Fermin Moscoso del Prado

University of Cambridge

I am a University Associate Professor of Computational Linguistics at the University of Cambridge (Department of Computer Science and Technology), and a Fellow of Jesus College Cambridge. Previously, I have held positions in Linguistics and in Psychology at UCSB, the French CNRS, and at the Medical Research Council. I hold an MEng in Computer Science by the Technical University of Madrid, and a PhD in LInguistics by Radboud University Nijmegen, obtained while working at the Max Planck Institute for Psycholinguistics.