Information and Coding Theory
Course ID: A8
Unit: DATA SCIENCE AND ENGINEERING – Unit A: Algorithms and Information Technologies
Weekly Hours: 4
ECTS Credits: 7
Description: Principles of Information Theory. Information Measures. Shannon’s theorem. Entropy, Relative Entropy and Mutual Information. Asymptotic Equipartition Property. Entropy Rates of a Stochastic Process. Probability of Error. Entropy Rate. Data Compression (Variable-length Lossless Compression, Fixed-length (almost lossless compression, Slepian-Wolf problem, universal compression). Differential Entropy. Gaussian Channel. Rate Distortion Theory. Information Theory and Statistics. Maximum Entropy. Universal Source Coding. Channel coding (Channel coding: Achievability bounds. Linear codes. Channel capacity. Channels with input constraints. Gaussian channels. Lattice codes, Lattice codes Channel coding: Energy-per-bit, continuous-time channels, Advanced channel coding. Source-Channel separation, Channel coding with feedback, Capacity-achieving codes via Forney concatenation). Kolmogorov Complexity. Network Information Theory. NP-hard coding theoretic problems. Applications in Complexity theory.