COMPSCI 650 (Applied Information Theory)


Arya Mazumdar


TuTh 16:00 - 17:15 Eng Lab 303

Office Hour

By Appointment


This course will introduce the basic concepts of Information Theory: entropy, relative entropy and mutual information. Applications of these concepts will be emphasized. The main topics will include information theory and statistics, inference, and learning methods.


No required textbook. The following references are useful.

  1. Thomas M. Cover, Joy A. Thomas, Elements of Information Theory, 2nd Edition, Wiley, Link.

  2. David MacKay, Information Theory, Inference, and Learning Algorithms, Cambridge. Download free HERE.

A previous installment of this course is available HERE

Grading Plan


Course Communication and Materials

We will use Moodle.