My Career Path and Research in Prosodic Entrainment in Dialogue

Abstract

In this talk I will first describe my own pretty unusual path from History to Computer Science as well as some things I learned along the way that might be useful for young researchers. I’ll next talk about a topic my students and I have worked on for many years -- entrainment: In conversation, speakers often adapt aspects of their speaking style to the style of their conversational partner. This phenomenon goes by many names, including entrainment, adaptation, and alignment. I will describe results from experiments on English and Mandarin prosodic entrainment in the Columbia Games Corpus and in the Tongji Games Corpus, large corpora of speech recorded from subjects playing a series of computer games. I will also discuss experiments relating entrainment to several social dimensions, including likeability and dominance, and its relationship to higher level prosodic features. This is joint work with Štefan Beňuš, Nishmar Cestero, Agustín Gravano, Rivka Levitan, Sarah Ita Levitan, and Zhihua Xia.

Biography

Julia Hirschberg is Percy K. and Vida L. W. Hudson Professor of Computer Science at Columbia University (department chair from 2012-2018). She previously worked at Bell Laboratories and AT&T Labs on text-to-speech synthesis (TTs) and then created the first HCI Research Department. She has served on the ACL executive board, the ISCA board (2005-7 as president), the CRA-WP board, the NAACL executive board, the CRA Executive Board, the AAAI Council, and the IEEE SLTC as well as numerous awards committees. She was editor of Computational Linguistics and Speech Communication and is a fellow of AAAI, ISCA, ACL, ACM, and IEEE, and a member of the NAE, the American Academy of Arts and Sciences, and the American Philosophical Society. She received the IEEE James L. Flanagan Speech and Audio Processing Award, the ISCA Medal for Scientific Achievement and the ISCA Special Service Medal. She has worked for diversity for many years at AT&T and Columbia. She studies speech and NLP, currently TTS, false information on social media and its intent, multimodal humor, radicalization in videos, and deceptive, trusted, emotional, empathetic, and charismatic speech.

The Use of Spoken Language Technologies for Screening Neurocognitive Disorders (NCD)

Abstract

This talk shares part of my journey in spoken language technologies research, starting from developing computer-aided pronunciation training technologies to support Chinese learners of English, to the development of AAC (augmentative and alternative communication) devices/technologies such as the e-communication book, dysarthric speech recognition, voice reconstruction, to our recent research in spoken language technologies for screening neurocognitive disorders (NCD). With the global population ageing rapidly, a key health concern lies in NCDs – a common form being Alzheimer’s Disease (AD). NCDs are particularly prominent in older adults, which has an insidious onset followed by gradual, irreversible deterioration in memory, communication, thinking and other cognitive functions. Thus, early detection of NCDs is crucial for timely intervention to slow down disease progression. This challenge motivated the formation of our interdisciplinary team, with researchers in engineering, geriatrics, linguistics, neurology, psychology and public health. We will present our recent research work in spoken language technologies to analyze English and Chinese recordings from neuropsychological assessments for discriminating between healthy controls and participants with NCD. Encouraging experimental results support the viability of fully automatic screening. We will also describe ongoing research on naturalistic data collection and spoken language interface development for older adult users.

Biography

Helen Meng is Patrick Huen Wing Ming Professor of the Department of Systems Engineering and Engineering Management, The Chinese University of Hong Kong. She received the S.B., S.M. and Ph.D. degrees, all in Electrical Engineering from MIT. She joined The Chinese University of Hong Kong in 1998, and established the Human-Computer Communications Laboratory in her department in 1999. In 2005, she founded the Microsoft-CUHK Joint Laboratory for Human-Centric Computing and Interface Technologies and serves as Director. This laboratory has been recognized as a Ministry of Education (MoE) of China Key Laboratory since 2008. In 2006, she founded the Tsinghua-CUHK Joint Research Centre for Media Sciences, Technologies and Systems. In 2007, she helped establish the Laboratory for Ambient Intelligence and Multimodal Systems in the Chinese Academy of Sciences Shenzhen Institute of Advanced Technology, through its joint initiative with CUHK. In 2013, she established the CUHK Stanley Ho Big Data Decision Analytics Research Center and serves as its Founding Director. Helen served as Associate Dean (Research) of the Faculty of Engineering between 2006 and 2010, as well as Department Chairman between 2012 and 2018.

Professor Meng is a recognized scholar in the field of multilingual speech and language processing, multimodal human-computer interaction and Big Data analytics. She leads the interdisciplinary research team that received the first Theme-based Research Scheme Project in Artificial Intelligence in 2019. She is a Fellow of the IEEE, elected in 2013 “for contributions to spoken language and multimodal systems”; a Fellow of the International Speech Communication Association (one of 62 worldwide, 8 from Asia), elected in 2016 “for contributions to multilingual, multimodal human-computer interaction and language learning technologies”; and a Fellow of the Hong Kong Institution of Engineers (HKIE) and Hong Kong Computer Society (HKCS). Her recent awards include 2018 CogInfoComm Best Paper Award, 2017 Outstanding Women Professional Award (one of 20 since 1999), 2016 Microsoft Research Outstanding Collaborator Award (one of 32 academics worldwide), 2016 IBM Faculty Award, 2016 IEEE ICME Best Paper Award, 2015 ISCA Distinguished Lecturer, 2015 HKCS inaugural Outstanding ICT Women Professional Award and 2012 Asia-Pacific Signal and Information Processing Association (APSIPA) inaugural Distinguished Lecturer.

Professor Meng devotes much effort towards professional services both regionally and internationally. She was elected Editor-in-Chief (2009-2011) of the IEEE Signal Processing Society’s (SPS) Transaction on Audio, Speech and Language Processing (often regarded as the most prestigious journal in the field), IEEE SPS Board of Governors (2014-2016), and currently serves on the Nominations and Appointments Committee (2017-2018). Helen was also an elected member of the International Speech Communication Association (ISCA) Board (2007-2015) and current serves on its International Advisory Council (2017-2020). She was Technical Chair of ISCA’s flagship conference INTERSPEECH 2014 and General Chair of INTERSPEECH 2020. She is the recipient of the 2019 IEEE Leo L. Beranek Meritorious Service Award for “for exemplary service to and leadership in the Signal Processing Society”.

Who Is Responsible for Responsible AI?

Abstract

With widespread application of intelligent systems in the society today, we are facing unprecedented demands to make signal processing and AI algorithms more robust & safe, more fair, more secure, more explainable and more accountable. In this talk, I will give an introduction to such challenges in our research field today and describe my personal experience in awakening to the need of making AI "responsible". I will discuss the role and impact of a signal processing researcher in the new era of Responsible AI.

Biography

Pascale Fung is a Professor at the Department of Electronic & Computer Engineering and Department of Computer Science & Engineering at The Hong Kong University of Science & Technology (HKUST), and a visiting professor at the Central Academy of Fine Arts in Beijing. She is an elected Fellow of the Association for the Advancement of Artificial Intelligence (AAAI) for her "significant contributions to the field of conversational AI and to the development of ethical AI principles and algorithms", an elected Fellow of the Association for Computational Linguistics (ACL) for her “significant contributions towards statistical NLP, comparable corpora, and building intelligent systems that can understand and empathize with humans”. She is an Fellow of the Institute of Electrical and Electronic Engineers (IEEE) for her “contributions to human-machine interactions” and an elected Fellow of the International Speech Communication Association for “fundamental contributions to the interdisciplinary area of spoken language human-machine interactions”. She is the Director of HKUST Centre for AI Research (CAiRE), an interdisciplinary research centre on top of all four schools at HKUST. She co-founded the Human Language Technology Center (HLTC). She is an affiliated faculty with the Robotics Institute and the Big Data Institute at HKUST. She is the founding chair of the Women Faculty Association at HKUST. She is an expert on the Global Future Council, a think tank for the World Economic Forum. She represents HKUST on Partnership on AI to Benefit People and Society. She is on the Board of Governors of the IEEE Signal Processing Society. She is a member of the IEEE Working Group to develop an IEEE standard - Recommended Practice for Organizational Governance of Artificial Intelligence. Her research team has won several best and outstanding paper awards at ACL, ACL and NeurIPS workshops.

Prof. Fung was born in Shanghai to professional artist parents but found her calling in artificial intelligence when she became interested in science fiction as a child. Today, her research interest lies in building intelligent systems that can understand and empathise with humans. To achieve this goal, her specific areas of research are using statistical modelling and deep learning for natural language processing, spoken language systems, emotion and sentiment recognition, and other areas of AI. She served as Editor and Associate Editor for Computer Speech and Language, IEEE/ACM Transactions on Audio, Speech and Language Processing, Transactions for ACL, IEEE Signal Processing Letters. She served as a Committee Member of the IEEE Signal Processing Society Speech and Language Technology Committee (SLTC) for six years. She is a past president t and a Board Member of the ACL Special Interest Group on Linguistics Data and Corpus Based Approaches in NLP (SIGDAT).

Prof. Fung received her PhD in Computer Science from Columbia University in 1997. She worked and studied at AT&T Bell Labs (1993~1997), BBN Systems & Technologies (1992), LIMSI, CNRS, France (1991),Department of Information Science, Kyoto University, Japan (1989~1991), and at Ecole Centrale Paris, France(1988). She speaks seven European and Asian languages.