The University of Sheffield
Neil Lawrence ML@SITraN
ICML 2008 Tutorial: Dimensionality Reduction the Probabilistic Way

ICML 2008 Tutorial: Dimensionality Reduction the Probabilistic Way

Neil D. Lawrence

Goals and Summary

The main focus of this tutorial will be probabilistic interpretations of dimensional reduction. It is aimed to complement the tutorial given by Lawrence Saul at NIPS 2005 on "Spectral Methods for Dimensional Reduction". Its particular focus will be probabilistic approaches to dimensional reduction based on generative models. These approaches have become increasingly popular in graphics and vision through the Gaussian Process Latent Variable Model. However, there also is a history to these methods which is perhaps less widely known amongst the newer generation of researchers. In particular the Generative Topographic Mapping and Latent Density Networks. This tutorial will give grounding to these methods through unifying them in the context of probabilistic latent variable models. This will involve a introduction to these approaches through the mechanism of probabilistic PCA, then a discussion of density networks leading into the generative topographic mapping. Finally the dual interpretation of probabilistic PCA and its extension to the GP-LVM will be given. Throughout the tutorial we will develop intuition about the methods with an ongoing set of example data sets. A particular focus of these example data sets will be motion capture data. Motion capture data is a nice example to use because it is easy for the human eye to tell when samples from the model are realistic.

One aspect of the tutorial will be the difference between the probabilistic approaches and the more commonly applied spectral approaches. In particular we will emphasize the distance preservation character of the probabilistic approaches: namely that local distances in the data are not necessarily preserved in the latent space. This contrasts with spectral algorithms which typically aim to preserve such local distances. These different characteristics mean that probabilistic approaches complement the spectral approaches, but the bring their own range of associated problems, in particular local minima in the optimization space. Heuristics for avoiding these local minima will also be discussed.

Slides are now available here.

Code is available from here.

Tutorial Outline

Motivation

Probabilistic Dimensionality Reduction

Spectral Methods and Probabilistic Methods Comparison

Density Networks and GTM

GP-LVM

Brief Bio of Speaker

Neil Lawrence is a Senior Research Fellow in the School of Computer Science at the University of Manchester, U.K.. Previous to this appointment he was a Senior Lecturer in the Department of Computer Science at the University of Sheffield, U.K. where he was head of the Machine Learning Research Group. His main research interest is machine learning through probabilistic models. He is interested in both the algorithmic side of these models and their application in areas such as computational systems biology, health, speech, vision and graphics.

His PhD was awarded in 2000 from the Computer Lab at the University of Cambridge. He then spent a year at Microsoft Research, Cambridge before moving to Sheffield in 2001 and then to Manchester in 2007.

This document last modified Thursday, 09-Oct-2014 19:42:56 BST