NUIM CS401 F2005

Mon 12:00-12:50 CS2
Tue 15:00-15:50 Hume6
Instructor: Barak Pearlmutter,
Office: Hamilton Institute (NUIM, Rye Hall, South Wing, room 5)
Office hours: you are welcome any time, just drop on by. (Afternoons are best, excepting every third Friday.) Or feel free to email or ring me up (x6394) and make an appointment.


Machine Learning, by Tom Mitchell.


  1. (19-Sep-2005) Intro to Machine Learning

  2. (20-Sep-2005) Perceptron Learning Rule (ch4 1/4)

  3. (26-Sep-2005) Linear and semilinear units (ch4 2/4)

  4. (27-Sep-2005) Convergence of LMS (ch4 3/4)

  5. (3-Oct-2005) Derivation of Backpropagation

  6. (4-Oct-2005) Lab: implementation of perceptron, backpropagation

  7. (10-Oct-2005) Generalization curves (ch4 4/4)

  8. (11-Oct-2005) Information Theory, Bayes' Rule, KL-Divergence, Entropy

  9. (17-Oct-2005) More Bayes' Rule, Evidence, Prior, Gaussian Distributions, Hypothesis Space, weight decay

  10. (18-Oct-2005) EM of mixture of iid binary vectors

  11. (24-Oct-2005) EM of mixture of Gaussians

  12. (25-Oct-2005) HMMs

  13. (7-Nov-2005) HMMs part duex

  14. (8-Nov-2005) HMMs part shalosh (w/ some practice)

  15. (14-Nov-2005) Support Vector Machines

  16. (15-Nov-2005) Support Vector Machines II

  17. (21-Nov-2005) Decision Trees I

  18. (22-Nov-2005) Decision Trees II

  19. (28-Nov-2005) VC Dimension

  20. (5-Dec-2005) PAC learning; strong learning vs weak learning; boosting and the ADAboost algorithm

  21. (6-Dec-2005) Multiplicative updates and irrelevant features (WINNOW)

  22. (12-Dec-2005) Review of Assignment 3; rants about modularity

  23. (13-Dec-2005) Review

Useful Materials

Author's lecture notes from the textbook: TM-book

Code written in class, usually scrubbed up a bit.


HMM Cheat Sheet


Pertinent answers to questions about the assignments will be added to the bottom of the assignment pages. Please check there before asking a question, as it might already be answered. But if it is not, please do ask - this will be of benefit not only to you but to everyone in the class.
  1. (19-Sep-2005) machine learning application brainstorm
  2. (4-Oct-2005) fun with perceptrons and backprop
  3. (28-Nov-2005) Battle HMM!
  4. (13-Dec-2005) take home practice quiz

Assignment-Related Warnings

  1. Late assignments will not be accepted.
  2. Don't cheat OR ELSE.
    (Talking about the assignments, and even getting some debugging help from your friends, is fine. Please give them credit though: "Thanks to Sally Helperman for assistance finding a mysterious bug in blah blah blah, and to Sam Sucker for spending seven hours showing me how to use Excel to generate ugly plots." And obviously copying, or not actually doing the work yourself, is not okay.)
  3. Sometimes I'll talk to people about what they turned in for an assignment. If you can't discuss your work cogently with me, I might be led to suspect that it was not in fact your own work.
  4. Even though assignments are only 30% of the grade, successful completion of the course is contingent upon satisfactory performance on the assignments. (In other words: if you don't do the assignments you don't pass the course even if you score 100% on exams.)