(Online) Words and the nature of artificial and human intelligence

Presenters: Janet B. Pierrehumbert

VT2MRC4

Is AI intelligent? Can state-of-the-art large language models (LLMs) pass the Turing Test? Part of the answer to this question depends on whether LLMs display human-like capabilities in learning, producing, and understanding words. No other animals share our ability to use words to exchange information about objects, events, and abstractions that are not directly observable. A large and adaptable lexicon is accordingly a hallmark of human intelligence. This workshop will systematically compare psycholinguistic results on the mental lexicon with the core properties of LLMs. It will distinguish large-scale memorization from generalization, and explain the mechanisms by which successful generalizations occur.

About half the workshop will cover successes of the current transformer-based LLMs. Like the original neural network models, current LLMs capture cumulative, gradient effects of similarity and frequency in predicting the acceptability or likelihood of novel forms. These effects appear in the cognitive literature under the rubric of “analogy” and “lexical gang effects”. Advancing beyond the original neural network models, LLMs can exploit similarities in meaning as well as in form, and they can learn from experience without direct feedback. They can activate and de-activate subfields of the vocabulary depending on the topic of discussion. The other half of the workshop will cover shortcomings of LLMs, concentrating on two problem areas. First, the LLMs do not bootstrap a mental lexicon in the same way that humans do. Second, although their treatment of word meaning and semantic similarity may work well for topical words, it breaks down for logical operators such as epistemic modals and sentential adverbs. These problems go towards explaining why LLMs often produce logically incoherent discourse. On the assumption that being logical defines the best of human intelligence, we will conclude that LLMs are not yet intelligent.

Keywords: Psycholinguistics, Learning, AI, Deep Learning, Morphology, Production, Productivity, Semantics

Presenters

Janet B. Pierrehumbert

University of Oxford

Janet B. Pierrehumbert is the Professor of Language Modelling in the University of Oxford Engineering Science Department. She holds degrees in Linguistics from Harvard and MIT. Much of her Ph.D thesis work was done in the Department of Linguistics and AI Research at AT&T Bell Labs, where she also served as a Member of Technical staff until 1989. She then took up a faculty position in Linguistics at Northwestern University, establishing an interdisciplinary research effort in experimental and computational linguistics. She is known for her research on prosody and intonation, as well as her work on how people acquire and use lexical systems that combine general abstract knowledge of word forms with detailed phonetic knowledge about individual words. In 2015, Pierrehumbert moved to her present position in the the Oxford e-Research Centre at Oxford, where she also holds a courtesy appointment in the Faculty of Linguistics, Phonetics, and Philology. Her lab group currently focusses on Natural Language Processing, emphasizing questions about the robustness and interpretability of language models and the dynamics of language in communities. She is a fellow of the LSA, the Cognitive Science Society, and the American Academy of Arts and Sciences. She was elected to the National Academy of Sciences in 2019, She was awarded the Medal for Scientific Achievement from the ISCA (the International Speech Communication Association) in 2020, and was elected as a member of the Academia Europaea in 2024.