CMPSCI 691GG • Fall 2006 • Tuesday and Thursday, 11:1512:30 • Agricultural Engineering Building (AEBC) S308 (See map)
Instructor 
Erik LearnedMiller 

Teaching Assistant: None  
Prerequisites 
Stats 515 or equivalent. Basic collegelevel probability. Basic programming ability in C++, Matlab, or Java. 

Reading Materials 
• Required text: Elements of Information Theory, Second Edition, by Thomas Cover and Joy Thomas.
• NOTE THAT THIS IS THE SECOND EDITION, NOT THE FIRST EDITION!!!
If you have a copy of the first edition of this book, you might be
able to use it, as the second edition is not too different. However,
you should at least have access to the second edition to check for
differences, page number references, problem set numbers, and other
differences. I will not answer questions about the differences between
the books, as I do not have time to figure this out. You should have
at least one friend with a copy of the second edition if you want to
use the first edition, so that you can find appropriate page number references and so on.


Resources 
Minimum entropy joint alignment David Mackay's Information Theory book (electronic edition is free and on the web) Guest Lecturer John Fisher's slides  
Problem Sets 
Problem sets are due at any time on the day indicated on the course
web page. If you turn them in after class, please slide them under the
door of my office, or put them in my faculty mailbox. The latest you
can turn in a problem set is midnight on the day due. The fact that
you cannot get into the building after hours does not count as an
excuse. If you're worried about that, turn it in earlier. I may take
off 50% for problem sets turned in late.  
Description 
This course will introduce the basic concepts of Information Theory: entropy, relative entropy, mutual information, channel capacity, and rate distortion. Applications, rather than proofs, will be emphasized. In addition, the statistical problem of computing information theoretic quantities from data will be emphasized. Nonparametric and semiparametric statistical models will also be covered.


Schedule 
