Week 7: Bayesian Regression

Lab Class

The notebook for the lab class can be downloaded from here.

To obtain the lab class in ipython notebook, first open the ipython notebook. Then paste the following code into the ipython notebook

import urllib
urllib.urlretrieve('https://raw.githubusercontent.com/SheffieldML/notebook/master/lab_classes/machine_learning/week7.ipynb', 'week7.ipynb')

You should now be able to find the lab class by clicking File->Open on the ipython notebook menu.

YouTube Videos

There is a YouTube video available of me giving this material at the Gaussian Process Road Show in Uganda.

GPRS Uganda Video

Second half overlaps with the material from this week's lectures.

Video from 2011 on Gaussian Densities and Bayesian Inference

Reading

Rogers and Girolami Chapter 3: Bayesian Methods Section 3.1-3.3 (pg 95-117) although you haven't covered the beta distribution.
Sections 1.2.3 (pg 21-24) of Bishop
Sections 1.2.6 (start from just past equ 1.64, pg 30-32) of Bishop
Section 2.3 of Bishop up to top of pg 85 (multivariate Gaussians).
Section 3.3 of Bishop up to pg 159 (pg 152-159). (Bayesian linear regression)
Sections 3.7-3.8 of Rogers and Girolami (pg 122-133).
Section 3.4 of Bishop (pg 161-165).

Previous Lectures

Univariate Bayesian Inference

Multivariate Bayesian Inference

Bayesian Polynomials on Olympics Data

Learning Outcomes Week 7

Be able to distinguish between different types of uncertainty: aleatoric and epistemic. Be able to give examples of each type.
Be able to derive Bayes rule' from the product rule of probability.
Understand the meaning of the terms prior, posterior and marginal likelihood
- Be able to identify these terms in Bayes' rule.
- Be able to describe what each of these terms represents (belief before observation, belief after observation, relationship between belief and observation, the model score.)
Understand how to derive the marginal likelihood from the likelihood and the prior.
Understand the difference between the frequentist approach and the Bayesian approach, i.e. that in the Bayesian approach parameters are treated as random variables
Be able to derive the maths to perform a simple Bayesian update on the offset parameter of a regression problem.

This document last modified Tuesday, 11-Nov-2014 08:10:41 UTC