ATIAM ML 2009

From IMTR

Instructor: Arshia Cont

This course provides an introduction to pattern recognition and statistical learning for the ATIAM Masters at Ircam during the school year 2009-2010. The goal of the course series is to expose students to problem solving tools using machine learning techniques and intuitions behind each approach.

Topics covered include: Bayesian decision theory; parameter estimation; maximum likelihood; Bayesian parameter estimation; conjugate and non-informative priors; dimensionality and dimensionality reduction; principal component analysis; linear discriminant analysis; density estimation: parametric vs. kernel-based methods; Nearest Neighbor methods; mixture models; expectation-maximization; Sequential Learning; HMMs; musical applications.

Contents 1 Resources 2 Grading 3 Lectures 4 Group Homeworks

Resources

• Duda, Hart and Stork, Pattern Classification, Wiley Interscience, 2000.
• C. Bishop, Pattern Recognition and Machine Learning, Springer, 2006.
• T. Hastie, R. Tibshirani, J. Friedman, Elements of Statistical learning, Springer, 2001. (Also available online)
• E. Alpaydin, Introduction to Machine Learning, MIT Press, 2004.
• L. Wasserman, All of Statistics: A concise course in statistical inference. Springer Verlag, 2006.

Grading

Grades for the ATIAM Machine Learning course will be distributed evenly between two grade options in ATIAM:

1. Group Grade: A grade based on reports of a TP (Travaux Pratique) which will be held on 15/12/2009 at 14h30 at ENST, including realization of a Matlab application following given instructions on a musical/machine learning problem.

• 20% of this grade comes from the Group Homework assignments given throughout the course.
• This grade will be distributed within the TSM option and constitues 30% of the overall grade (or 6 over 20).

2. A written exam on 22/12/2009 at 10h, lasting one hour.

• This grade will be distributed evenly with the final STIM grade (50%)

Lectures

Date	Topic	Material	Size
09/11/2009	Introduction, Bayesian decision theory	slides	(~10Mb)
16/11/2009	Maximum Likelihood, Bayesian Parameter Estimation Group Homework 1	slides	(~6.8Mb)
30/11/2009	Bayesian Parameter Estimation, Priors and Conjugates, Kernel Densities, K-NN, Mixture Densities, Basics of EM Group Homework 2 C. Bishop's slides on Probability Distributions	slides	(~10Mb)
01/12/2009	EM Algorithm, Clustering, Sequential Learning, HMMs and variants Group Homework 3 & 4 Intro to Lagrange Multipliers	slides	(~7.8Mb)
07/12/2009	Linear Discriminant Learning, Gradient Descent, Intro to SVM, Large Margin Classifiers	slides	(~4.3Mb)

Group Homeworks

Due Date	Topic	Set	Size
30/11/2009	Maximum Likelihood on Polynomial Regression	PDF	(~400kb)
7/12/2009	Bayesian Parameter Estimation on Multinomial/histogram problems	PDF	(~650kb)
7/12/2009	EM derivation for mixtures of exponential distributions	PDF	(~182kb)
7/12/2009	HMM State Duration Modeling	PDF	(~295kb)