Machine Learning at U of C

This page contains a list of topics, definitions, and results from Machine Learning course at University of Chicago.

Week 1

Learning problem

Given a distribution $\mathbb {P}$ on $X\times Y$. We want to learn the objective function $f_{p}(x)=\mathbb {E} _{p}[y|x]$ (with respect to the distribution $\mathbb {P}$).

Learning Algorithms

Let Z be the set of possible samples. The learning algorithm is a function that maps a number of samples to a measurable function (denoted here by F a class of all measurable functions). Sometimes we consider a class of computable functions instead.

$A:\cup _{n=1}^{\infty }Z^{n}\rightarrow F$

Loss function

Suppose the learning algorithm outputs h. The learning error can be measured by

$\int (y-h(x))^{2}dP$

One can prove that minimizing this quantity could be reduced to the problem of minimizing the following quantity.

$||f_{p}-h||_{l_{2}(\mathbb {P} )}^{2}=\int (f_{p}(x)-h(x))^{2}P_{x}(x)dx$

And that's the reason why we try to learn $\mathbb {E} _{p}[y|x]$