MINDLAB READING
This page is created to be the (temporary?) main information page of the MIND lab reading group. Contents in our reading group can be any old or new (but interesting or important to many members) research works.
เนื้อหา
Plan
For the first 2-3 week, I, Jung, will take the lead. I will cover three topics which are frequently mentioned in literatures and I'm most familiar with: (1) Mixture Models and EM algorithm (2) Sampling methods (3) Principal Component Analysis and regularization.
After that, if it successes, I hope that we can continue the process and change the leader week by week. The leader should have at least 2 weeks to read the topic.
Date and Time
At this moment, I plan to arrange the reading group in every friday afternoon (1pm - 2pm), but let us talk for more convenient dates and times.
May 11, 2007: Mixture Models and EM algorithm
This topic is a nice introduction for Bayesian paradigm in machine learning. After this talk, we should be able to answer the following questions:
- What is the Bayesian machine learning paradigm?
- The Bayes equation
- Why being Bayesian is a good idea? What are advantages of the Bayesian paradigm over the classical paradigm?
- It is intuitive, easy to understand (but might not easy to do)
- It can solve the model selection problem
- How can we train the learner in Bayesian paradigm?
- This talk illustrates one Bayesian toolkit: the EM algorithm.
Main paper
- Chris Bishop's PRML book chapter 9. ( I have one copy left; anyone who do not have this book please feel free to borrow me)
Supplementary
- David Mackay's book chapters 2 (prerequisite), 20 and 22 get it here
- Zoubin Ghahramani's slide lecture2 lecture3 (DO NOT read PCA and FA)
- Chris Bishop's slide click (DO NOT read variational inference)
Prerequisite
In order to understand how EM work, we have to understand a bit on Information entropy, the Jensen's inequality and Kullback-Leibler divergence. So please take a look at its properties before reading group: David Mackay's book chapter 2 is a very good introduction.
May 18, 2007: The Monte Carlo Methods#1
Last talk illustrate one way to do Bayesian inference: the EM algorithm. However, it can give us only the maximum likelihood of the parameter which is not we exactly want in Bayesian inference. This talk will introduce more general Bayesian toolkit's namely, the monte carlo methods.
The topics we will cover are rejection sampling, uniform sampling, importance sampling, and introduction to Markov chain.
