IEEE ICASSP 2022

The proposed presentation is essentially an overview of continual lifelong learning for (deep) neural network based machine learning systems, with a focus on conversational artificial intelligence.

It is structured in three parts: one, the evolution of continual lifelong learning in the context of deep learning and takeaways that inspire the present; two, the current state-of-the art in continual lifelong learning; and three, the challenges with the present techniques and where the current research to address those challenges is headed. Throughout all the three parts, insights will be mapped to spoken language understanding and conversational AI systems in general.

Most machine learning based products today employ copious usage of deep learning based systems across wide variety of domains and various modes of signals - visual, aural, linguistic, gestural and spatial. However, most of the deployed systems are also typically trained on batches of stationary data without accounting for the possible changes in incoming signals in data, and thus suffer from regression or a certain degree of catastrophic forgetting on encountering signal data drifts. Continual lifelong learning enables reduction in effort on retraining such systems by enabling generalization skills in intelligent systems; the current and future research in this field will therefore lead to such systems functioning with higher autonomy than they currently do.

In this talk, we will cover how this paradigm has evolved thus far, and how it applies to conversational AI; dialog systems in particular, that are based on spoken language understanding and typically consist of aural and linguistic signals in data. We will also cover current state of the art - systems that employ transfer learning based approaches and multimodal learning based approaches toward achieving continual lifelong learning and research in the last couple of years (2020), which exhibits focus in understanding the tradeoffs between generalization and alleviating catastrophic forgetting, characterizing the latter that has led to novel neural network structures. These structures are yet to be effectively and widely productized. We will go over these approaches in the talk as well, particularly placing emphasis on networks that employ regularization-based plasticity (to penalize forgetting older signals) and experience replays (to integration information from past in periodic episodes during training phase).

Lastly, we will cover some insights from recent research as to where current challenges lie and where the next few years will take us with respect to novel neural network architectures. This is especially of fascination and importance to the signal processing community - both in academia and industry - to discover, improve and integrate more biological aspects of lifelong learning exhibited in mammalian brains, like multisensory integration for example, into evolving artificial intelligent and autonomous systems.

Biography

Pooja received her Bachelor's degree from Ramaiah Institute of Technology, Bangalore, India in Telecommunications Engineering (2013) and M.S degree in Electrical Engineering from University of Southern California (2015), where she specialized in vision-based robotics and machine learning. She has held research intern roles at USC Institute for Creative Technologies, USC Robotic Embedded Systems Laboratory, and Nvidia, where she worked on projects related to robot navigation and 3D scanning hardware. Since 2015, she has been a part of the AI/ML software industry in Silicon Valley - developing and delivering Conversational AI based products. As part of Knowles Intelligent Audio, she shipped Keyword-Spotting systems for consumer mobile phones utilizing Hidden Markov Models and Deep Neural Networks. As part of Cisco, she developed deep learning based keyword spotting systems for Webex voice assistant deployed in enterprise meeting devices, and shipped call control and notification features for Cisco 730 series smart bluetooth headsets. Currently, as a part of IBM Data and AI, she’s working at the intersection of speech recognition, natural language understanding and software to help bring intelligent conversational experiences to Quick Service Restaurants like McDonald's drive-thrus. Outside of her day job, Pooja is a part of World Economic Forum's Global Shapers initiative, and works with the Palo Alto chapter to help the local community prepare for ramifications of the fourth industrial revolution, currently focussing on social impact in education through bridging digital divide.