| 3/30/04 | Introduction/history; basic information measures - entropy, relative entropy, mutual information. | [slides] |
| 4/1/04 | Convexity, Jensen's inequality and consequences, convexity properties of entropy and mutual information. | [Notes] |
| 4/6/04 | The Data Processing Inequality, Fano's Inequality, the Asymptotic Equipartition Property. | [Notes] |
| 4/8/04 | Typical sets, Shannon's Source Coding theorem and converse, Entropy rate of a stochastic process. | [Notes] |
| 4/13/04 | Entropy rate of a stochastic process, Shannon-McMillan-Breiman Theorem, Markov Chains. | [Notes] |
| 4/15/04 | Lossless source codes, uniquely decodable codes, instantaneous codes, Kraft inequality, optimal codes. | |
| 4/20/04 | Kraft inequality for uniquely decodable codes, Shannon codes, Huffman Codes. | |
| 4/22/04 | Optimality of Huffman Codes, universal source codes, Lempel-Ziv Compression | [Notes] |
| 4/27/04 | Introduction to channel coding, discrete-time/discrete alphabet channels, discrete memoryless channels, information capacity. | |
| 4/29/04 | Calculating capacity- symmetric channels, convex optimization; Joint typicality | |
| 5/4/04 | MID-TERM EXAM. | |
| 5/7/04 | Joint typicality; block codes, random coding argument. | |
| 5/11/04 | Proof of the direct part of channel coding theorem for DMC; expurgated codes; implementation issues. | |
| 5/13/04 | Strong Coding theorems, error exponents and reliability functions; Converse to the coding theorem. | [Notes] |
| 5/18/04 | Converse to the coding theorem for bit error probability, the joint source channel coding theorem, feedback capacity | |
| 5/20/04 | Feedback capacity, continuous-valued random variables and differential entropy. | |
| 5/25/04 | More on Differential entropy, Gaussian RV's, capacity of the discrete-time additive white Gaussian noise channel. | |
| 5/27/04 | Coding theorem and converse for additive white Gausian noise channel, High SNR and low SNR regimes, parallel Gaussian channels and water-filling. | |
| 6/1/04 | Continuous-time (waveform) channel models, sampling and equivalent discrete-time channels, colored Gaussian noise channels. | |
| 6/3/04 | Rate-Distortion theory, quantization, Lloyd-Max algorithm, vector quantizers, rate-distortion function, Gaussian sources |