My Career Path and Research in Prosodic Entrainment in Dialogue

Abstract

In this talk I will first describe my own pretty unusual path from History to Computer Science as well as some things I learned along the way that might be useful for young researchers. I’ll next talk about a topic my students and I have worked on for many years -- entrainment: In conversation, speakers often adapt aspects of their speaking style to the style of their conversational partner. This phenomenon goes by many names, including entrainment, adaptation, and alignment. I will describe results from experiments on English and Mandarin prosodic entrainment in the Columbia Games Corpus and in the Tongji Games Corpus, large corpora of speech recorded from subjects playing a series of computer games. I will also discuss experiments relating entrainment to several social dimensions, including likeability and dominance, and its relationship to higher level prosodic features. This is joint work with Štefan Beňuš, Nishmar Cestero, Agustín Gravano, Rivka Levitan, Sarah Ita Levitan, and Zhihua Xia.

Biography

Julia Hirschberg is Percy K. and Vida L. W. Hudson Professor of Computer Science at Columbia University (department chair from 2012-2018). She previously worked at Bell Laboratories and AT&T Labs on text-to-speech synthesis (TTs) and then created the first HCI Research Department. She has served on the ACL executive board, the ISCA board (2005-7 as president), the CRA-WP board, the NAACL executive board, the CRA Executive Board, the AAAI Council, and the IEEE SLTC as well as numerous awards committees. She was editor of Computational Linguistics and Speech Communication and is a fellow of AAAI, ISCA, ACL, ACM, and IEEE, and a member of the NAE, the American Academy of Arts and Sciences, and the American Philosophical Society. She received the IEEE James L. Flanagan Speech and Audio Processing Award, the ISCA Medal for Scientific Achievement and the ISCA Special Service Medal. She has worked for diversity for many years at AT&T and Columbia. She studies speech and NLP, currently TTS, false information on social media and its intent, multimodal humor, radicalization in videos, and deceptive, trusted, emotional, empathetic, and charismatic speech.