Theory of Machine Learning

The Theory of Machine Learning group studies fundamental questions in learning theory and statistics, informed by perspectives grounded in algorithms, computational complexity, statistics, information theory, probability, statistical physics, and more. We are interested in questions such as:

• What are the best algorithms for extracting useful information from large, noisy, and high-dimensional datasets?
• What are the mathematical structures in data which enable computationally-efficient learning?
• Why do deep neural networks work so well for challenging classification and prediction tasks?
• Which learning problems are fundamentally intractable, and why?

We aim both to explain the astonishing success of today’s machine learning methods and to provide new algorithmic tools to jumpstart new machine learning paradigms, while building and maintaining sound mathematical foundations for learning. MIT researchers have helped to shape this emerging field, with contributions across the following topics (and more):

• High-dimensional statistics and probability: how do we learn from images, genomes, and other inherently high-dimensional data?
• Robust learning: how can algorithms maintain accuracy in the face of data which has been adversarially corrupted or otherwise does not fit the expected distribution?
• Reinforcement learning: how can we learn and make decisions in changing environments?
• Sub-linear learning: how can we get useful information from datasets so large that even storing or looking at the entire dataset is prohibitively costly?
• Dimension reduction: how can we find useful low-dimensional structure in high-dimensional data?
• Learning from non-i.i.d. data: how can we learn when independence assumptions are violated – what if data is provided by selfish agents?
• Complexity of statistics: what are the fundamental limits to computationally efficient learning, both on its own and with additional desirable properties such as robustness and privacy, and how can we overcome these limits?

Affiliated Faculty:

Guy Bresler

Phillippe Rigollet

Elchannan Mossel

Sasha Rakhlin

Yury Polyanskiy

Nike Sun

Martin Wainwright

David Gamarnik