ECE 547 / CS 591.04: Neural Networks


Your guides in the discovery of your inner nature are:

Course Materials

Computing Resources


My approach to teaching this sort of material is quite practical: I'd rather have you prove your mastery of something by implementing it or doing experiments with it than by answering some questions about it. Although there will be in-class quizzes, there will also be a strong programming and experimenal assignment component to the class.

Everyone in the class will do a final project (typically in teams of two). Neural networks are my research area, and I am very happy to help students who become particularly excited about some topic learn more about it.

Course materials including assignments will be posted on the web. (You can get extra credit by writing up html lecture notes in a timely fashion for me to post on the web.)

Topics to be covered


You will need to

I'm not a stickler about formal prereqs - effort can make up for a lot. But I expect you to not be scared of working through equations when they come up, or thinking in mathematical terms. Let's say ``mathematical training or appropriate mathematical maturity.''


This syllabus will be revised as the semester progresses. Links to lecture notes, keys for the quizzes and problems sets, and other germane material will be added appropriate points below.

The book is much more signal-processing oriented than this course. You are not obligated to understand material unrelated to the lectures.

TueAug 25 historical background
ThuAug 27 my idiosyncratic view of neural networks (ch 2)
TueSep 1 perceptrons (ch 3) assignment
ThuSep 3 in-class demo of perceptron learning
TueSep 8 perceptrons: histograms, Widrow's LMS, multiplicative updates
ThuSep 10 perceptrons: theoretical limitations, randomization
TueSep 15 multilayer perceptrons and backpropagation of error (ch 4)
ThuSep 17 more backpropagation, assignment
TueSep 22 Kolmogorov's theorem; debugging gradient optimization
ThuSep 24 autoassociative networks and dimmensionality reduction
TueSep 29 overfitting and generalization
ThuOct 1 more generalization
TueOct 6 VC dimension, assignment
ThuOct 8 PAC learning, VC dimension of MLPs
TueOct 13 convergence rate of gradient descent
ThuOct 15 (Fall break)
TueOct 20 The Bayesian approach to generalization
ThuOct 22 (continued) assignment
TueOct 27 Gaussian mixture models, EM, LVQ
ThuOct 29 Hierarchical Mixtures of Experts
TueNov 3 Boltzmann machines
ThuNov 5 Fixedpoint backpropagation, MFT Boltzmann take-home quiz
TueNov 10 (Doug Eck) Reinforcement learning
ThuNov 12 (Doug Eck) (continued)
TueNov 17 (Doug Eck) Q-learning
ThuNov 19 Helmholtz machines
TueNov 24 Blind source separation
ThuNov 26 (Thanksgiving)
TueDec 1 (Michael Zibulevsky) Support Vector Machines
ThuDec 3 (Doug Eck) Recurrent Nets
TueDec 8 Backpropagation through time, RTRL
ThuDec 10 final project presentations
ThuDec 17 5pm, final exam due

Barak Pearlmutter <>