All lecture notes (except slides) in one PDF: All lecture notes
| Topic | Contents | Links | |
| 1. | Introduction to Machine Learning | Overview of topics |
Machine Learning (Wikipedia) |
| 2. | Linear Regression | 1D regression Multidiensional regression |
MATLAB transcript (Sep 17) Linear Regresion (Wikipedia) |
| 3. | Nonlinear Regression | Radial Basis Functions Neural Networks K-nearest neighbors |
RBFs (Wikipedia) ANNs (Wikipedia) KNN (Wikipedia) |
| 4. | Gradient Descent | Gradient descent Line Search |
Gradient descent (Wikipedia) Line Search (Wikipedia) Optimization (Wikipedia) |
| 5. | Cross Validation | Hold-out Validation N-Fold Cross Validation |
Cross-validation (Wikipedia) |
| 6. | Classification | Class conditionals Logistic Regression Neural nets for classification Naïve Bayes |
Logistic Regression (Wikipedia) Naïve Bayes (Wikipedia) |
| 7. | Bayesian Methods | Bayesian Regression Relationship to Maximum Likelihood Regression Hyperparameters Model Selection |
Bayesian model selection demos (Tom Minka) |
| 8. | Monte Carlo Methods | Sampling Gaussians Importance Sampling Metropolis Hastings |
MCMC (Wikipedia) MCMC applet |
| 9. | Principal Components Analysis | PCA Whitening PPCA |
PCA (Wikipedia) |
| 10. | Clustering | K-means Mixtures of Gaussians Expectation-Maximization |
K-means (Wikipedia) |
| 11. | Hidden Markov Models | Markov chains Viterbi Forward-Backward Baum-Welch (EM) |
HMMs (Wikipedia) |
| 12. | Lagrange Multipliers | Equality constraints Bounds constraints |
Lagrange Multipliers (Wikipedia) |
| 13. | Support Vector Machines | Maximum margin Loss functions Kernels |
SVMs (Wikipedia) |
| 14. | AdaBoost | Boosting Exponential loss Early stopping |
Tutorial slides on AdaBoost |