Dr. Sriram Ganapathy

Self supervision and large language models - The Buzz of Generative AI

The revolution in AI caused by deep learning has allowed the development of a diverse set of techniques that are applicable to a range of domains like text, speech, images and beyond. While this advancement is itself fascinating to discuss, the recent innovations of the past 3 years have given rise to a new field of artificial intelligence, called self-supervised learning. Self-supervision, a method of unsupervised learning, is branch of machine learning where the models learn directly from data without the need of exhaustive labels. Underlying these models are deep sequence-to-sequence models called transformers. The early success stories of self-supervision have already caused substantial disruption in the filed in the last few months, a.k.a the openAI's chatGPT and Google's Bard.

This talk will dive into the principles of self-supervision and its application to large language models (LLMs). I will attempt to introduce the problem construction and illustrate the modeling capabilities with examples from fields like natural language processing (NLP) and speech processing. The extensions to other domains and tasks will be discussed. I will also describe the implications of such models in various fields such as science and education as well as the ethical considerations in the design.

The talk is aimed to be basic and will attempt to open up a discussion on the various aspects of these models.