# ATIAM ML 2009

### From IMTR

**Instructor:** Arshia Cont

This course provides an introduction to pattern recognition and statistical learning for the ATIAM Masters at Ircam during the school year 2009-2010. The goal of the course series is to expose students to problem solving tools using machine learning techniques and intuitions behind each approach.

Topics covered include: Bayesian decision theory; parameter estimation; maximum likelihood; Bayesian parameter estimation; conjugate and non-informative priors; dimensionality and dimensionality reduction; principal component analysis; linear discriminant analysis; density estimation: parametric vs. kernel-based methods; Nearest Neighbor methods; mixture models; expectation-maximization; Sequential Learning; HMMs; musical applications.

## Contents |

### Resources

- Duda, Hart and Stork,
*Pattern Classification*, Wiley Interscience, 2000. - C. Bishop,
*Pattern Recognition and Machine Learning*, Springer, 2006. - T. Hastie, R. Tibshirani, J. Friedman,
*Elements of Statistical learning*, Springer, 2001. (Also available online) - E. Alpaydin,
*Introduction to Machine Learning*, MIT Press, 2004. - L. Wasserman,
*All of Statistics: A concise course in statistical inference.*Springer Verlag, 2006.

### Grading

Grades for the ATIAM Machine Learning course will be distributed evenly between two grade options in ATIAM:

1. *Group Grade*: A grade based on reports of a TP (Travaux Pratique) which will be held on 15/12/2009 at 14h30 at ENST, including realization of a Matlab application following given instructions on a musical/machine learning problem.

- 20% of this grade comes from the
*Group Homework*assignments given throughout the course. - This grade will be distributed within the TSM option and constitues 30% of the overall grade (or 6 over 20).

- 20% of this grade comes from the

2. A written exam on 22/12/2009 at 10h, lasting one hour.

- This grade will be distributed evenly with the final STIM grade (50%)

### Lectures

Date | Topic | Material | Size |
---|---|---|---|

09/11/2009 | Introduction, Bayesian decision theory | slides | (~10Mb) |

16/11/2009 | Maximum Likelihood, Bayesian Parameter Estimation
- Group Homework 1
| slides | (~6.8Mb) |

30/11/2009 | Bayesian Parameter Estimation, Priors and Conjugates, Kernel Densities, K-NN, Mixture Densities, Basics of EM
- Group Homework 2
- C. Bishop's slides on Probability Distributions
| slides | (~10Mb) |

01/12/2009 | EM Algorithm, Clustering, Sequential Learning, HMMs and variants
- Group Homework 3 & 4
- Intro to Lagrange Multipliers
| slides | (~7.8Mb) |

07/12/2009 | Linear Discriminant Learning, Gradient Descent, Intro to SVM, Large Margin Classifiers | slides | (~4.3Mb) |

### Group Homeworks

Due Date | Topic | Set | Size |
---|---|---|---|

30/11/2009 | Maximum Likelihood on Polynomial Regression | (~400kb) | |

7/12/2009 | Bayesian Parameter Estimation on Multinomial/histogram problems | (~650kb) | |

7/12/2009 | EM derivation for mixtures of exponential distributions | (~182kb) | |

7/12/2009 | HMM State Duration Modeling | (~295kb) |